Weather-related disruptions have a significant impact on a variety of industries, including agriculture, infrastructure, and public safety. Predicting these unusual weather events remains a significant challenge. The problem is complicated by the lack of high-quality weather data due to the failure of sensors at weather stations during severe weather. In this work, we proposed a novel method to classify rare severe weather-related events by incorporating publicly available tweets with the meteorological conditions readings collected from weather stations across Alaska. The use of multimodal data of varying quality is introduced to compensate for missing meteorological recordings obtained from the weather stations. In our study, we collected geotagged tweets from the region of focus and utilized context-aware BERT embeddings to rigorously analyze and ensure the validity and dependability of the social media texts. Labels for the social sensor data were generated based on weather events associated with the tweet collection. For predicting rare weather events, we proposed a multiclass classification model. This machine learning model was trained and tested using data from the year 2020. The results obtained by learning from the integrated data showed a significant increase in the F1 score when compared to relying on weather data alone. Our findings indicate that a model supplemented with daily weather and social media text data outperforms alternatives enhanced with hourly data. The proposed model achieved an F1-score of 0.83 for multimodal data, compared to 0.30 obtained by the baseline model that relies solely on weather data. Training the proposed model with the combined dataset significantly improved performance, resulting in a 95% accuracy.

This work investigates how the rich semantic embeddings of pre-trained language models can be used to help understand the general attitudes of an online community. This work describes a novel prediction model that can ingest statements describing an arbitrary context and a piece of content, and output answers to a set of 'attitude questions' describing the relationship between them. Typically, annotating answers to questions like "Does this contain sarcasm?", or "Is this content positive with respect to this context?" requires costly human interaction. In this work, we consider the ability of large language models to answer these questions, while under the constraint of a small dataset using a novel prediction head. We show that this methodology can accurately answer these attitude questions, compare the model to off-the-shelf language model approaches, and describe a method for collecting and annotating attitude question data sets. The novel attitude question answering model achieves a 89% accuracy on the attitude question answering task, outperforming the ablated models (87%) as well as the off the shelf models using BERT-based Sequence Classification (13%), BART-based Natural Language Inference (88%), and RoBERTa-based Question-Answering (87%).

With the increase in volume of daily online news items, it is more and more difficult for readers to identify news articles relevant to their interests. Thus, effective recommendation systems are critical for an effective user news consumption experience. Existing news recommendation methods usually rely on the news click history to model user interest. However, there are other signals about user behaviors, such as user commenting activity, which have not been used before. We propose a recommendation algorithm that predicts articles a user may be interested in, given her historical sequential commenting behavior on news articles. We show that following this sequential user behavior the news recommendation problem falls into in the class of session-based recommendation. The techniques in this class seek to model users' sequential and temporal behaviors. While we seek to follow the general directions in this space, we face unique challenges specific to news in modeling temporal dynamics, e.g., users' interests shift over time, users comment irregularly on articles, and articles are perishable items with limited lifespans. We propose a recency-regularized neural attentive framework for session-based news recommendation. The proposed method is able to capture the temporal dynamics of both users and news articles, while maintaining interpretability. We design a lag-aware attention and a recency regularization to model the time effect of news articles and comments. We conduct extensive empirical studies on 3 real-world news datasets to demonstrate the effectiveness of our method.

Research on classifying time-series based on subsequences, known as shapelets, has attracted considerable interest in the community. Most existing shapelet-based time-series classification approaches neglect the temporal dependencies among extracted shapelets. Recently, shapelet-orders that encode the temporal dependencies among pairwise shapelets were shown to be informative features. However, based on a random selection of candidate shapelets, the state-of-the-art model does not guarantee optimal shapelets selection. This, in turn, may lead to inferior quality shapelet-orders. Learning shapelets, instead of searching, guarantees near-optimal shapelets thus decreasing generalization error. However, the costly initialization approach for learning generalized shapelets significantly limits its scalability in large time-series datasets. We address the problem of leveraging temporal dependencies among generalized shapelets from randomly initialized subsequences by jointly learning from the shapelet-transform space and the shapelet-order space. The underlying hypothesis is that leveraging the temporal dependency information of generalized shapelets improves the classification performance. Furthermore, introducing a randomized subsequence initialization for learning generalized shapelets allows a more scalable shapelet learning approach. The proposed model was significantly more accurate and faster than the baseline alternatives when evaluated on both synthetic and real-world time-series datasets.

The question of graph similarity or graph distinguishability arises often in natural systems and their analysis over graphical networks. In many domains, graph similarity is used for graph classification, outlier detection or the identification of distinguished interaction patterns. Several methods have been proposed on how to address this topic, but graph comparison still presents many challenges. Recently, information physics has emerged as a promising theoretical foundation for complex networks. In many applications, it has been demonstrated that natural complex systems exhibit features that can be described and interpreted by measures typically applied in quantum mechanical systems. Therefore, a natural starting point for the identification of network similarity measures is information physics and a series of measures of distance for quantum states. In this work, we report experiments on synthetic and real-world data sets, and compare quantum-inspired measures to a series of state-of-the-art and well-established methods of graph distinguishability. We show that quantum-inspired methods satisfy the mathematical and intuitive requirements for graph similarities, while offering high interpretability.

Most of the natural systems encountered in all kinds of disciplines consist of a set of elementary units connected by relationships of different kinds. These complex systems are commonly described in terms of networks, where nodes represent the entities and links represent their interactions. As multiple types of distinct interactions are often observed, these systems are described as multiplex networks where the different types of interactions between the nodes constitute the different layers of the network. The ever-increasing size of these networks introduces new computational challenges and is therefore imperative to be able to eliminate the redundant or irrelevant edges of a network and create a summary that maintains the intrinsic properties of the original network, with respect to the overall structure of the system. In this work, we present a summarization technique for multiplex networks designed to maintain the structural characteristics of such complex systems by utilizing the intrinsic multiplex structure of the network and taking into consideration the inter-connectivity of the various graph layers. We validate our approach on real-world systems from different domains and show that our approach allows for the creation of more compact summaries, with minimum change of the structure evaluation measures, when compared to baseline methods that aggregate contributions of multiple types of interactions.

Digital media companies typically collect rich data in the form of sequences of online user activities. Such data is used in various applications, involving tasks ranging from click or conversion prediction to recommendation or user segmentation. Nonetheless, each application depends upon specialized feature engineering that requires a lot of effort and typically disregards the time-varying nature of the online user behavior. Learning time-preserving vector representations of users (user embeddings), irrespective of a specific task, would save redundant effort and potentially lead to higher embedding quality. To that end, we address the limitations of the current state-of-the-art self-supervised methods for task-independent (unsupervised) sequence embedding, and propose a novel Time-Aware Sequential Autoencoder (TASA) that accounts for the temporal aspects of sequences of activities. The generated embeddings are intended to be readily accessible for many problem formulations and seamlessly applicable to desired tasks, thus sidestepping the burden of task-driven feature engineering. The proposed TASA shows improvements over alternative self-supervised models in terms of sequence reconstruction. Moreover, the embeddings generated by TASA yield increases in predictive performance on both proprietary and public data. It also achieves comparable results to supervised approaches that are trained on individual tasks separately and require substantially more computational effort. TASA has been incorporated within a pipeline designed to provide time-aware user embeddings as a service, and the use of its embeddings exhibited lifts in conversion prediction AUC on four audiences.

In this study, a procedure is proposed for surveying public opinion from big social media domain-specific textual data to minimize the difficulties associated with modeling public behavior. Strategies for labeling posts relevant to a topic are discussed. A two-part framework is proposed in which semi-automatic labeling is applied to a small subset of posts, referred to as the "seed" in further text. This seed is used as bases for semi-supervised labeling of the rest of the data. The hypothesis is that the proposed method will achieve better labeling performance than existing classification models when applied to small amounts of labeled data. The seed is labeled using posts of users with a known and consistent view on the topic. A semi-supervised multi-class prediction model labels the remaining data iteratively. In each iteration, it adds context-label pairs to the training set if softmax-based label probabilities are above the threshold. The proposed method is characterized on four datasets by comparison to the three popular text modeling algorithms (n-grams + tfidf, fastText, VDCNN) for different sizes of labeled seeds (5,000 and 50,000 posts) and for several label-prediction significance thresholds. Our proposed semi-supervised method outperformed alternative algorithms by capturing additional contexts from the unlabeled data. The accuracy of the algorithm was increasing by (3-10%) when using a larger fraction of data as the seed. For the smaller seed, lower label probability threshold was clearly a better choice, while for larger seeds no predominant threshold was observed. The proposed framework, using fastText library for efficient text classification and representation learning, achieved the best results for a smaller seed, while VDCNN wrapped in the proposed framework achieved the best results for the bigger seed. The performance was negatively influenced by the number of classes. Finally, the model was applied to characterize a biased dataset of opinions related to gun control/rights advocacy. The proposed semi-automatic seed labeling is used to label 8,448 twitter posts of 171 advocates for guns control/rights. On this application, our approach performed better than existing models and it achieves 96.5% accuracy and 0.68 F1 score.

Temporal graph regression is a frequently encountered research problem in many studies of graph analytics. A temporal graph is a sequence of attributed graphs where node features and target variables change over time, but network structure stays constant. The task of temporal graph regression is to predict the target variables associated with nodes at future time-points given historical snapshots of the graph. Existing methods tackle this problem mostly by conducting structured regression for all target variables. However, those methods have limited performance due to redundant information. Although several techniques have been proposed recently to learn lower dimensional embedding for the target space, the problem of how to effectively exploit the structure of the temporal graph in such embeddings is still unsolved. Other recent works only study node embedding of the stationary graphs only, and this is not applicable to temporal attributed graphs. In this paper, we introduced a Structure-Aware Intrinsic Representation Learning model (SAIRL) to jointly learn lower dimensional embeddings of the target space and feature space via structure-aware graph abstraction and feature-aware target embedding learning. To solve this problem, we have developed a derivative-free block coordinate descent algorithm with closed-form solutions. To characterize the quality of embedding-based learned with SAIRL, we conducted extensive experiments on a variety of different real-world temporal graphs. The results indicate that the proposed method can be more accurate than the state-of-the-art embedding learning methods, regardless of regressors.

Highly discriminative short time-series subsequences, known as shapelets, are used to classify a time-series. The existing shapelet-based methods for time-series classification assume that shapelets are independent of each other. However, they neglect temporal dependencies among pairs of shapelets, which are informative features that exist in many applications. Within this new framework, we explore a scheme to extract informative orders among shapelets by considering the time gap between two shapelets. In addition, we propose a novel model, Pairwise Shapelet-Orders Discovery, which extracts both informative shapelets and shapelet-orders and incorporates the shapelet-transformed space with shapelet-order space for time-series classification. The hypothesis of the study is that the extracted orders could increase the confidence of the prediction and further improves the classification performance. The results of extensive experiments conducted on 75 univariate and 6 multivariate real-world datasets provide evidence that the proposed model could significantly improve accuracy on average over baseline methods.

Optimization of patient throughput and wait time in emergency departments (ED) is an important task for hospital systems. For that reason, Emergency Severity Index (ESI) system for patient triage was introduced to help guide manual estimation of acuity levels, which is used by nurses to rank the patients and organize hospital resources. However, despite improvements that it brought to managing medical resources, such triage system greatly depends on nurse's subjective judgment and is thus prone to human errors. Here, we propose a novel deep model based on the word attention mechanism designed for predicting a number of resources an ED patient would need. Our approach incorporates routinely available continuous and nominal (structured) data with medical text (unstructured) data, including patient's chief complaint, past medical history, medication list, and nurse assessment collected for 338,500 ED visits over three years in a large urban hospital. Using both structured and unstructured data, the proposed approach achieves the AUC of ∼ 88% for the task of identifying resource intensive patients (binary classification), and the accuracy of ∼ 44% for predicting exact category of number of resources (multi-class classification task), giving an estimated lift over nurses' performance by 16% in accuracy. Furthermore, the attention mechanism of the proposed model provides interpretability by assigning attention scores for nurses' notes which is crucial for decision making and implementation of such approaches in the real systems working on human health.

Rapid expansion of mobile devices has brought an unprecedented opportunity for mobile operators and content publishers to reach many users at any point in time. Understanding usage patterns of mobile applications (apps) is an integral task that precedes advertising efforts of providing relevant recommendations to users. However, this task can be very arduous due to the unstructured nature of app data, with sparseness in available information. This study proposes a novel approach to learn representations of mobile user actions using Deep Memory Networks. We validate the proposed approach on millions of app usage sessions built from large scale feeds of mobile app events and mobile purchase receipts. The empirical study demonstrates that the proposed approach performed better compared to several competitive baselines in terms of recommendation precision quality. To the best of our knowledge this is the first study analyzing app usage patterns for purchase recommendation.

Recovering network connectivity structure from high-dimensional observations is of increasing importance in statistical learning applications. A prominent approach is to learn a Sparse Gaussian Markov Random Field by optimizing regularized maximum likelihood, where the sparsity is induced by imposing L1 norm on the entries of a precision matrix. In this article, we shed light on an alternative objective, where instead of precision, its Cholesky factor is penalized by the L1 norm. We show that such an objective criterion possesses attractive properties that allowed us to develop a very fast Scale-Free Networks Estimation Through Cholesky factorization (SNETCH) optimization algorithm based on coordinate descent, which is highly parallelizable and can exploit an active set approach. The approach is particularly suited for problems with structures that allow sparse Cholesky factor, an important example being scale-free networks. Evaluation on synthetically generated examples and high-impact applications from a biomedical domain of up to more than 900,000 variables provides evidence that for such tasks the SNETCH algorithm can learn the underlying structure more accurately, and an order of magnitude faster than state-of-the-art approaches based on the L1 penalized precision matrix.

Structured regression has been successfully used in many applications where explanatory and response variables are inter-correlated, such as in weighted attributed networks. One of structured models, Gaussian Conditional Random Fields (GCRF), utilizing multiple unstructured models to learn the non-linear relationships between node attributes and the structured response variable, achieves high prediction accuracy. However, it does not scale well with large networks. We propose a novel model, called Scalable Approximate GCRF (SA-GCRF), which integrates weighted attributed network compression with GCRF, with the aim of making GCRF applicable to large networks. The model consists of three steps: first, it compresses a network into a smaller one by generalizing nodes into supernodes and edges into superedges; then, it applies GCRF to the reduced network; and finally, it unfolds the predicted response variables into the original nodes. Our hypothesis is that the reduced network maintains most information of the original network such that the loss in prediction accuracy obtained by GCRF on the reduced network is minor. The comprehensive experimental results indicate that SA-GCRF was 150-520 times faster than standard GCRF and 11-29 times faster than state-of-the-art UmGCRF on large networks, and provided regression results where GCRF and UmGCRF were not applicable. Furthermore, SA-GCRF achieved a similar regression accuracy, 0.76, to the one obtained from the original real-world weighted attributed citation network, even after compressing the network to 10% of its size.

It is of high interest for a company to identify customers expected to bring the largest profit in the upcoming period. Knowing as much as possible about each customer is crucial for such predictions. However, their demographic data, preferences, and other information that might be useful for building loyalty programs is often missing. Additionally, modeling relations among different customers as a network can be beneficial for predictions at an individual level, as similar customers tend to have similar purchasing patterns. We address this problem by proposing a robust framework for structured regression on deficient data in evolving networks with a supervised representation learning based on neural features embedding. The new method is compared to several unstructured and structured alternatives for predicting customer behavior (e.g. purchasing frequency and customer ticket) on user networks generated from customer databases of two companies from different industries. The obtained results show 4% to 130% improvement in accuracy over alternatives when all customer information is known. Additionally, the robustness of our method is demonstrated when up to 80% of demographic information was missing where it was up to several folds more accurate as compared to alternatives that are either ignoring cases with missing values or learn their feature representation in an unsupervised manner.

In many applications, including climate science, power systems, and remote sensing, multiple input variables are observed for each output variable and the output variables are dependent. Several methods have been proposed to improve prediction by learning the conditional distribution of the output variables. However, when the relationship between the raw features and the outputs is nonlinear, the existing methods cannot capture both the nonlinearity and the underlying structure well. In this study, we propose a structured model containing hidden variables, which are nonlinear functions of inputs and which are linearly related with the output variables. The parameters modeling the relationships between the input and hidden variables, between the hidden and output variables, as well as among the output variables are learned simultaneously. To demonstrate the effectiveness of our proposed method, we conducted extensive experiments on eight synthetic datasets and three real-world challenging datasets: forecasting wind power, forecasting solar energy, and forecasting precipitation over U.S. The proposed method was more accurate than state-of-the-art structured regression methods.

Until recently, research in social network analysis was focused on single layer networks, where only one type of links among nodes is considered. This approach does not consider the variety of interactions that exist among nodes, resulting in the loss of a large amount of information. In the last few years there is an advanced interest in multilayer networks analysis, where multiple types of nodal connections are considered jointly. In most approaches however the contributions of the various interactions are averaged, resulting again in the loss of information. In this work we present a structured regression model for node attribute prediction in multilayer networks. Our Gaussian Conditional Random Fields model is designed to maximize the information gained from the use of data with multiple layers of graphical structure. Our model accommodates graphs with layers that share the same set of nodes allowing for missing nodes and unobserved connections. At the same time it models the evolution of such networks over time without requiring the addition of a new layer. We present evidence that this model outperforms the traditionally used one and that it offers predictive accuracy that increases as the number of layers used grows, on both synthetic data and challenging real world applications such as predicting citation count and sepsis hospitalization admission rate at all hospitals in California.

The availability of big data [James et al. 2011; Steve 2012] and the emergence of wearable computing [Thad 1996; Alex 2000], network science [Barabasi 2002], and computational social science [Hanna 2016; Watts and Strogatz 1998] as areas of inquiry has been revolutionizing the landscape of how we decipher our lives, our social interactions, and our day-to-day activities.This well-connected world has promised novel requirements on transforming healthcare from reactive and hospital-centered, to preventive, proactive, evidence-based, person-centered, and focused on well-being rather than ailment recovery.A multitude of various types of data are involved in this broad context of healthcare, including the following:-Clinical data [Prather et al. 1997; Riccardo and Zupan 2008], mainly the patient records from clinical institutions, such as medical imaging, patient electronic health records, clinical trial data, etc. -Genotype data [Eibe et al. 2004; Leslie et al. 1999], basically the genetic makeups of the individuals, such as DNA, protein, etc. -Social media data [Reza et al. 2014; Sitaram and Huberman 2010], which is the information the individuals posted on online social platforms such as Facebook, Twitter, PatientsLikeMe, etc. -Environmental sensory data [Ruchi and Bhatia 2010], which is the information sampled from the surrounding environment where the individuals are living in, such as air pollution and humidity information.-Behavioral and sentiment data [Bo and Lee 2008], which could be the data recorded by the wearable devices on patient's activities.-Mobile data [Miller and Han 2009; Fosca and Pedreschi 2008], which is sampled from individuals' mobile devices.Integrating multiple types of information to make people healthier is also a problem of vital importance that requires collective effort from different parties, where data mining plays a pivotal role.Toward this aim, the National Science Foundation of United

Conditional probabilistic graphical models provide a powerful framework for structured regression in spatio-temporal datasets with complex correlation patterns. However, in real-life applications a large fraction of observations is often missing, which can severely limit the representational power of these models. In this paper we propose a Marginalized Gaussian Conditional Random Fields (m-GCRF) structured regression model for dealing with missing labels in partially observed temporal attributed graphs. This method is aimed at learning with both labeled and unlabeled parts and effectively predicting future values in a graph. The method is even capable of learning from nodes for which the response variable is never observed in history, which poses problems for many state-of-the-art models that can handle missing data. The proposed model is characterized for various missingness mechanisms on 500 synthetic graphs. The benefits of the new method are also demonstrated on a challenging application for predicting precipitation based on partial observations of climate variables in a temporal graph that spans the entire continental US. We also show that the method can be useful for optimizing the costs of data collection in climate applications via active reduction of the number of weather stations to consider. In experiments on these real-world and synthetic datasets we show that the proposed model is consistently more accurate than alternative semi-supervised structured models, as well as models that either use imputation to deal with missing values or simply ignore them altogether.

Early classification of time series is prevalent in many time-sensitive applications such as, but not limited to, early warning of disease outcome and early warning of crisis in stock market. \textcolor{black}{ For example,} early diagnosis allows physicians to design appropriate therapeutic strategies at early stages of diseases. However, practical adaptation of early classification of time series requires an easy to understand explanation (interpretability) and a measure of confidence of the prediction results (uncertainty estimates). These two aspects were not jointly addressed in previous time series early classification studies, such that a difficult choice of selecting one of these aspects is required. In this study, we propose a simple and yet effective method to provide uncertainty estimates for an interpretable early classification method. The question we address here is "how to provide estimates of uncertainty in regard to interpretable early prediction." In our extensive evaluation on twenty time series datasets we showed that the proposed method has several advantages over the state-of-the-art method that provides reliability estimates in early classification. Namely, the proposed method is more effective than the state-of-the-art method, is simple to implement, and provides interpretable results.

When faced with the task of forming predictions for nodes in a social network, it can be quite difficult to decide which of the available connections among nodes should be used for the best results. This problem is further exacerbated when temporal information is available, prompting the question of whether this information should be aggregated or not, and if not, which portions of it should be used. With this challenge in mind, we propose a novel utilization of variograms for selecting potentially useful relationship types, whose merits are then evaluated using a Gaussian Conditional Random Field model for node attribute prediction of temporal social networks with a multigraph structure. Our flexible model allows for measuring many kinds of relationships between nodes in the network that evolve over time, as well as using those relationships to augment the outputs of various unstructured predictors to further improve performance. The experimental results exhibit the effectiveness of using particular relationships to boost performance of unstructured predictors, show that using other relationships could actually impede performance, and also indicate that while variograms alone are not necessarily sufficient to identify a useful relationship, they greatly help in removing obviously useless measures, and can be combined with intuition to identify the optimal relationships.

Leveraging temporal observations to predict a patient's health state at a future period is a very challenging task. Providing such a prediction early and accurately allows for designing a more successful treatment that starts before a disease completely develops. Information for this kind of early diagnosis could be extracted by use of temporal data mining methods for handling complex multivariate time series. However, physicians usually prefer to use interpretable models that can be easily explained, rather than relying on more complex black-box approaches. In this study, a temporal data mining method is proposed for extracting interpretable patterns from multivariate time series data, which can be used to assist in providing interpretable early diagnosis. The problem is formulated as an optimization based binary classification task addressed in three steps. First, the time series data is transformed into a binary matrix representation suitable for application of classification methods. Second, a novel convex-concave optimization problem is defined to extract multivariate patterns from the constructed binary matrix. Then, a mixed integer discrete optimization formulation is provided to reduce the dimensionality and extract interpretable multivariate patterns. Finally, those interpretable multivariate patterns are used for early classification in challenging clinical applications. In the conducted experiments on two human viral infection datasets and a larger myocardial infarction dataset, the proposed method was more accurate and provided classifications earlier than three alternative state-of-the-art methods.

Identifying informative biomarkers from a large pool of candidates is the key step for accurate prediction of an individual's health status. In clinical applications traditional static feature selection methods that flatten the temporal data cannot be directly applied since the patient's observed clinical condition is a temporal multivariate time series where different variables can capture various stages of temporal change in the patient's health status. In this study, in order to identify informative genes in temporal microarray data, a margin based feature selection filter is proposed. The proposed method is based on well-established machine learning techniques without any assumptions about the data distribution. The objective function of temporal margin-based feature selection is defined to maximize each subject's temporal margin in its own relevant subspace. In the objective function, the uncertainty in calculating nearest neighbors is taken into account by considering the change in feature weights in each iteration. A fixed-point gradient descent method is proposed to solve the formulated objective function. The experimental results on both synthetic and real data provide evidence that the proposed method can identify more informative features than the alternatives that flatten the temporal data in advance.

In a data-mining approach, a model for estimation of aerosol optical depth (AOD) from satellite observations is learned using collocated satellite and ground-based observations. For accurate learning of such a spatio-temporal model, it is important to collect ground-based data from a large number of sites. The objective of this project is to determine appropriate locations for the next set of ground-based data collection sites to maximize accuracy of AOD estimation. Ideally, a new site should capture the most significant unseen aerosol patterns and should be the least correlated with the previously observed patterns. We propose achieving this aim by selecting the locations on which the existing prediction model is the most uncertain. Several criteria were considered for site selection, including uncertainty, spatial diversity, similarity in temporal pattern, and their combination. Extensive experiments on globally distributed data over 90 AERONET sites from the years 2005 and 2006 provide strong evidence that sites selected using the proposed algorithms improve the overall AOD prediction accuracy at a faster rate than those selected randomly or based on spatial diversity among sites.

In supervised learning, on data collected over space and time, different relationships can be found over different spatio-temporal regions. In such situations, an appropriate spatio-temporal data partitioning followed by building specialized predictors could often achieve higher overall prediction accuracy than when learning a single predictor on all the data. In practice, partitions are typically decided based on prior knowledge. As an alternative to domain-based partitioning, we propose a method that automatically discovers a spatio-temporal partitioning through the competition of regression models. The method is evaluated on a challenging problem using satellite observations to predict Aerosol Optical Depth (AOD), which represents the amount of depletion that a beam of radiation undergoes as it passes through the atmosphere. Our experiments used more than 20,000 labeled data points collected during 3 years from more than 100 sites worldwide. Our partitioning-based approach was compared to the recently developed operational AOD prediction algorithm, called C5, which uses domain knowledge for spatio-temporal partitioning of the Earth and implements a region-specific deterministic predictor that utilizes forward simulations from the postulated physical models. Data partitioning used in C5 divides the world into three spatio-temporal regions that differ based on the location and the time of the year as decided by domain experts. The results showed that a neural network predictor trained on all the data has accuracy comparable to C5. When specialized neural network predictors were learned on C5-based partitions, the overall prediction accuracy was not improved. On the other hand, our competition-based spatio-temporal data partitioning approach resulted in large accuracy improvements. The most accurate results were obtained when (1) the data from each of the sites were split into two temporal subsets, one for winter-spring months and another for summer-fall months; and (2) two neural network predictors were competing for each of the identified spatio-temporal subsets.

Aerosols are small airborne particles that both reflect and absorb incoming solar radiation and whose effect on the Earth's radiation budget is one of the biggest challenges of current climate research. To help address this challenge, numerous satellite sensors are employed to achieve global-scale monitoring of aerosols. Given the satellite measurements, the common objective is prediction of Aerosol Optical Depth (AOD). An important property of AOD is its low spatial variability on a scale of tens of kilometers. On the other hand, satellite sensors gather information in the form of multi-spectral images with high spatial resolution where pixels could be as small as a few hundred meters. Given an accurate ground-based AOD measurement over a specific location and time, all the pixels in the vicinity can be assumed to have the same AOD. If we treat satellite measurement at a single pixel as an instance, all pixels from the neighborhood can be considered as a bag of instances labeled with the same AOD. Given a number of bags obtained at numerous locations and at different times we can treat the problem of AOD prediction from satellite attributes as Multiple Instance Regression (MIR). An important challenge is that because of rapidly changing surface properties attribute values of pixels from a bag can vary a lot. This study evaluated several MIR approaches on several synthetic data sets and on a data set consisting of 800 labeled bags, each containing hundreds of pixel instances observed over the Continental U.S. by the MISR satellite instrument. The results indicate that the most successful MIR approach consists of an iterative procedure that detects and discards outlying instances and trains a predictor on the remaining ones.

In this study we proposed an iterative procedure for correcting sampling bias in labeled datasets for supervised learning applications. Given a much larger and unbiased unlabeled dataset, our approach relies on training contrast classifiers to iteratively select unlabeled examples most highly underrepresented in the labeled dataset. Once labeled, these examples could greatly reduce the sampling bias present in the labeled dataset. Unlike active learning methods, the actual labeling is not necessary in order to determine the most appropriate sampling schedule. The proposed procedure was applied on an important bioinformatics problem of prioritizing protein targets for structural genomics projects. We show that the procedure is capable of identifying protein targets that are underrepresented in current protein structure database, the Protein Data Bank (PDB). We argue that these proteins should be given higher priorities for experimental structural characterization to achieve faster sampling bias reduction in current PDB and make it more representative of the protein space.

Predictive data mining typically relies on labeled data without exploiting a much larger amount of available unlabeled data. We show that using unlabeled data can be beneficial in a range of important prediction problems and therefore should be an integral part of the learning process. Given an unlabeled dataset representative of the underlying distribution and a K-class labeled sample that might be biased, our approach is to learn K contrast classifiers each trained to discriminate a certain class of labeled data from the unlabeled population. We illustrate that contrast classifiers can be useful in one-class classification, outlier detection, density estimation, and learning from biased data. The advantages of the proposed approach are demonstrated by an extensive evaluation on synthetic data followed by real-life bioinformatics applications for (1) ranking PubMed articles by their relevance to protein disorder and (2) cost-effective enlargement of a disordered protein database.

A labeled sequence data set related to a certain biological property is often biased and, therefore, does not completely capture its diversity in nature. To reduce this sampling bias problem a data mining procedure is proposed for detecting underrepresented relevant sequences. The procedure is aimed at helping domain experts achieve a cost-effective qualitative enlargement of knowledge through an in-depth study of a small number of statistically underrepresented and functionally interesting sequences. Our procedure consists of: (i) learning a class-conditional distribution model on each class of labeled data; (ii) applying the models to select statistically underrepresented unlabeled sequences; and (iii) automatically evaluating their interestingness. An application of the proposed approach is illustrated on an important problem of increasing the data set of confirmed disordered proteins. The obtained results demonstrate the promise of the proposed approach for an efficient reduction of sampling bias in biological databases.

In this paper, we propose a general framework for distributed boosting intended for efficient integrating specialized classifiers learned over very large and distributed homogeneous databases that cannot be merged at a single location. Our distributed boosting algorithm can also be used as a parallel classification technique, where a massive database that cannot fit into main computer memory is partitioned into disjoint subsets for a more efficient analysis. In the proposed method, at each boosting round the classifiers are first learned from disjoint datasets and then exchanged amongst the sites. Finally the classifiers are combined into a weighted voting ensemble on each disjoint data set. The ensemble that is applied to an unseen test set represents an ensemble of ensembles built on all distributed sites. In experiments performed on four large data sets the proposed distributed boosting method achieved classification accuracy comparable or even slightly better than the standard boosting algorithm while requiring less memory and less computational time. In addition, the communication overhead of the distributed boosting algorithm is very small making it a viable alternative to the standard boosting for large-scale databases.

Combining multiple global models (e.g. back-propagation based neural networks) is an effective technique for improving classification accuracy. This technique reduces variance by manipulating the distribution of the training data. In many large scale data analysis problems involving heterogeneous databases with attribute instability, standard boosting methods can be improved by coalescing multiple classifiers. Each classifier uses different germane attribute information that is identified through the attribute selection process. We propose a new technique of boosting localized classifiers when heterogeneous data sets contain more homogeneous data distributions. Instead of a single global classifier for each boosting round, we have localized classifiers responsible for each homogeneous region. The number of regions is identified through a clustering algorithm performed at each boosting iteration. A new boosting method applied to real life spatial data and synthetic spatial data shows improvements in prediction accuracy when unstable driving attributes and heterogeneity are present in the data. In addition, boosting localized experts significantly reduces the number of iterations needed for achieving the maximal prediction accuracy.

1 Introduction Prediction of a continuous response variable in spatial-temporal domains has recently drawn attention in a data analysis community [4],[11]. Spatial-temporal regression models, learned on systematically collected values of driving attributes, can contribute to better understanding of complex phenomena studied in meteorology, oceanography, environmental science, precision agriculture and other domains. However, spatial-temporal modelling is often difficult due to various factors. For example, due to a prohibitive cost of data collection or other constraints, it is often not possible to systematically measure values of all attributes that have an influence to the observed response [15]. In such applications, models estimated on available attributes often have unsatisfactory explanatory power.

No abstract available.