A central question in the era of big data is what to do with the enormous amount of information. One possibility is to characterize it through statistics, e.g., averages, or classify it using machine learning, in order to understand the general structure of the overall data. The perspective in this paper is the opposite, namely that most of the value in the information in some applications is in the parts that deviate from the average, that are unusual, atypical. We define what we mean by atypical in an axiomatic way as data that can be encoded with fewer bits in itself rather than using the code for the typical data. We show that this definition has good theoretical properties. We then develop an implementation based on universal source coding, and apply this to a number of real world data sets.

With the enormous amount of data generated through the internet and sensors, Internet of Things, it becomes too overwhelming for humans to examine it all. One solution is to reduce the data to a set of statistics. The perspective in this paper is the opposite, namely that most of this data is just background noise, and the interesting parts are those that deviate from background noise, the parts that are atypical. In order to find such “interesting” parts of data, universal approaches are required, since it is not known in advance what we are looking for. The paper develops new algorithms for detecting atypical data based on information theory concepts. These are applied to a number of real-world data.

One characteristic of the information age is the exponential growth of information, and the ready availability of this information through networks, including the internet - “Big Data.” The question is what to do with this enormous amount of information. One possibility is to characterize it through statistics - think averages. The perspective in this paper is the opposite, namely that most of the value in the information is in the parts that deviate from the average, that are unusual, atypical. Think of art: the valuable paintings or writings are those that deviate from the norms, that are atypical. The same could be true for venture development and scientific research. The paper first discusses what exactly should be understood by “atypical.” This is by no means straightforward. It has to be a well defined theoretical concept corresponding to some intuitive idea of atypicality, which when applied gives useful results. This is followed by a simple example of iid binary sequences. This example is simple enough that complete algorithms can be developed and analyzed, which give insights into atypicality. We finally develop a more general algorithm based on the Context Tree Weighing algorithm and apply that to heart rate variability.