Sparse representation has been widely used in data compression, signal and image denoising, dimensionality reduction and computer vision. While overcomplete dictionaries are required for sparse representation of multidimensional data, orthogonal bases represent one-dimensional data well. In this paper, we propose a data-driven sparse representation using orthonormal bases under the lossless compression constraint. We show that imposing such constraint under the Minimum Description Length (MDL) principle leads to a unique and optimal sparse representation for one-dimensional data, which results in discriminative features useful for data discovery.

The task of predicting the interactions between drugs and targets plays a key role in the process of drug discovery. There is a need to develop novel and efficient prediction approaches in order to avoid costly and laborious yet not-always-deterministic experiments to determine drug-target interactions (DTIs) by experiments alone. These approaches should be capable of identifying the potential DTIs in a timely manner. In this article, we describe the data required for the task of DTI prediction followed by a comprehensive catalog consisting of machine learning methods and databases, which have been proposed and utilized to predict DTIs. The advantages and disadvantages of each set of methods are also briefly discussed. Lastly, the challenges one may face in prediction of DTI using machine learning approaches are highlighted and we conclude by shedding some lights on important future research directions.

The first version of this article neglected to acknowledge Maryam Bagherian and Elyas Sabeti's equal contributions to the study. This has now been corrected. The publisher regrets the error.

Acute respiratory distress syndrome (ARDS) is a fulminant inflammatory lung injury that develops in patients with critical illnesses, affecting 200,000 patients in the United States annually. However, a recent study suggests that most patients with ARDS are diagnosed late or missed completely and fail to receive life-saving treatments. This is primarily due to the dependency of current diagnosis criteria on chest x-ray, which is not necessarily available at the time of diagnosis. In machine learning, such an information is known as Privileged Information - information that is available at training but not at testing. However, in diagnosing ARDS, privileged information (chest x-rays) are sometimes only available for a portion of the training data. To address this issue, the Learning Using Partially Available Privileged Information (LUPAPI) paradigm is proposed. As there are multiple ways to incorporate partially available privileged information, three models built on classical SVM are described. Another complexity of diagnosing ARDS is the uncertainty in clinical interpretation of chest x-rays. To address this, the LUPAPI framework is then extended to incorporate label uncertainty, resulting in a novel and comprehensive machine learning paradigm - Learning Using Label Uncertainty and Partially Available Privileged Information (LULUPAPI). The proposed frameworks use Electronic Health Record (EHR) data as regular information, chest x-rays as partially available privileged information, and clinicians' confidence levels in ARDS diagnosis as a measure of label uncertainty. Experiments on an ARDS dataset demonstrate that both the LUPAPI and LULUPAPI models outperform SVM, with LULUPAPI performing better than LUPAPI.

While Internet of Things (IoT) devices and sensors create continuous streams of information, Big Data infrastructures are deemed to handle the influx of data in realtime. One type of such a continuous stream of information is time series data. Due to the richness of information in time series and inadequacy of summary statistics to encapsulate structures and patterns in such data, development of new approaches to learn time series is of interest. In this paper, we propose a novel method, called pattern tree, to learn patterns in the times-series using a binary-structured tree. While a pattern tree can be used for many purposes such as lossless compression, prediction and anomaly detection, in this paper we focus on its application in time series estimation and forecasting. In comparison to other methods, our proposed pattern tree method improves the mean squared error of estimation.

A central question in the era of big data is what to do with the enormous amount of information. One possibility is to characterize it through statistics, e.g., averages, or classify it using machine learning, in order to understand the general structure of the overall data. The perspective in this paper is the opposite, namely that most of the value in the information in some applications is in the parts that deviate from the average, that are unusual, atypical. We define what we mean by atypical in an axiomatic way as data that can be encoded with fewer bits in itself rather than using the code for the typical data. We show that this definition has good theoretical properties. We then develop an implementation based on universal source coding, and apply this to a number of real world data sets.

With the enormous amount of data generated through the internet and sensors, Internet of Things, it becomes too overwhelming for humans to examine it all. One solution is to reduce the data to a set of statistics. The perspective in this paper is the opposite, namely that most of this data is just background noise, and the interesting parts are those that deviate from background noise, the parts that are atypical. In order to find such “interesting” parts of data, universal approaches are required, since it is not known in advance what we are looking for. Our approach is to use Rissanen's minimum description length (MDL) as a tool for that. We would like to be able to find both short and long atypical sequences of data, and we therefore need accurate expressions of MDL, without prior assumptions. In this paper we develop a modified predictive MDL method that works better for short sequences.

With the enormous amount of data generated through the internet and sensors, Internet of Things, it becomes too overwhelming for humans to examine it all. One solution is to reduce the data to a set of statistics. The perspective in this paper is the opposite, namely that most of this data is just background noise, and the interesting parts are those that deviate from background noise, the parts that are atypical. In order to find such “interesting” parts of data, universal approaches are required, since it is not known in advance what we are looking for. The paper develops new algorithms for detecting atypical data based on information theory concepts. These are applied to a number of real-world data.

With the exponential growth in information, “Big Data,” a key question is what to do with this information. A classic possibility is to characterize it through statistics. The perspective in this paper is the opposite, namely that most of the value in the information is in the parts that deviates from the average, that are unusual, atypical. Think of art: The valuable paintings or writings are those that deviate from the norms, that are atypical. With the same perspective think of social networks: Flickr is an example that has a rating for images based on how interesting they are. In previous works, researchers introduced algorithms to find the interesting images based on human psychological taste; but in this paper, we are proposing a new method to rate an image based on their level of interestingness using atypicality, in which interestingness is equivalent to having atypical structure. The paper first discusses what exactly should be understood by “atypical.” It has to be a well defined theoretical concept corresponding to some intuitive idea of atypicality, which when applied gives useful results. This is followed by applying the notion of atypicality to the general discrete scenario with finite alphabet and then putting it in to the framework of the quantized wavelet image coding. Finally our proposed algorithm is applied on Labeled Faces in the Wild (LFW) database with 13,233 images in order to find the interesting ones.

Atypical sequences are subsequences of long sequences that deviates from the ‘normal’ data. In a previous paper we have developed an information theory approach to such sequences for discrete data. In the current paper we extend this principle to real-valued data, whereby it is possible to use signal processing tools to search for atypical data. The application of this principle is to extract a few interesting sets of information from ‘big data’ sets. We include a simple application to stock market data.

One characteristic of the information age is the exponential growth of information, and the ready availability of this information through networks, including the internet - “Big Data.” The question is what to do with this enormous amount of information. One possibility is to characterize it through statistics - think averages. The perspective in this paper is the opposite, namely that most of the value in the information is in the parts that deviate from the average, that are unusual, atypical. Think of art: the valuable paintings or writings are those that deviate from the norms, that are atypical. The same could be true for venture development and scientific research. The paper first discusses what exactly should be understood by “atypical.” This is by no means straightforward. It has to be a well defined theoretical concept corresponding to some intuitive idea of atypicality, which when applied gives useful results. This is followed by a simple example of iid binary sequences. This example is simple enough that complete algorithms can be developed and analyzed, which give insights into atypicality. We finally develop a more general algorithm based on the Context Tree Weighing algorithm and apply that to heart rate variability.