Inducing adaptive predictive models in real-time from high throughput data streams is one of the most challenging areas of Big Data Analytics. The fact that data streams may contain concept drifts (changes of the pattern encoded in the stream over time) and are unbounded, imposes unique challenges in comparison with predictive data mining from batch data. Several real-time predictive data stream algorithms exist, however, most approaches are not naturally parallel and thus limited in their scalability. This paper highlights the Micro-Cluster Nearest Neighbour (MC-NN) data stream classifier. MC-NN is based on statistical summaries of the data stream and a nearest neighbour approach, which makes MC-NN naturally parallel. In its serial version MC-NN is able to handle data streams, the data does not need to reside in memory and is processed incrementally. MC-NN is also able to adapt to concept drifts. This paper provides an empirical study on the serial algorithm’s speed, adaptivity and accuracy. Furthermore, this paper discusses the new parallel implementation of MC-NN, its parallel properties and provides an empirical scalability study.

Mining data streams is a core element of Big Data Analytics. It represents the velocity of large datasets, which is one of the four aspects of Big Data, the other three being volume, variety and veracity. As data streams in, models are constructed using data mining techniques tailored towards continuous and fast model update. The Hoeffding Inequality has been among the most successful approaches in learning theory for data streams. In this context, it is typically used to provide a statistical bound for the number of examples needed in each step of an incremental learning process. It has been applied to both classification and clustering problems. Despite the success of the Hoeffding Tree classifier and other data stream mining methods, such models fall short of explaining how their results (i.e., classifications) are reached (black boxing). The expressiveness of decision models in data streams is an area of research that has attracted less attention, despite its paramount of practical importance. In this paper, we address this issue, adopting Hoeffding Inequality as an upper bound to build decision rules which can help decision makers with informed predictions (white boxing). We termed our novel method Hoeffding Rules with respect to the use of the Hoeffding Inequality in the method, for estimating whether an induced rule from a smaller sample would be of the same quality as a rule induced from a larger sample. The new method brings in a number of novel contributions including handling uncertainty through abstaining, dealing with continuous data through Gaussian statistical modelling, and an experimentally proven fast algorithm. We conducted a thorough experimental study using benchmark datasets, showing the efficiency and expressiveness of the proposed technique when compared with the state-of-the-art.

CDR (Call Detail Records) data is more easily available than other network related data (such as GPS data) as most telecommunications service providers (TSPs) maintain such data. By analyzing it one can find mobility patterns of most of the population thus leading to efficient urban planning, disease and traffic control, etc. But its granularity is low as the latitude and longitude (lat-lon) of a cell tower is used as the current location of all mobile phones that are connected to the cell tower at that time. Granularity can range between 10s of metres to several kms depending on population density of a locality. This is one reason why, although there are many existing systems on visualizing mobility of people based on GPS data, there is hardly any existing system for CDR. We develop a Mobility Visualization System (MoVis) for visualizing mobility of people from their CDR records. First of all, given the CDR records of a user, we determine her stay regions (places where she stays for a significant amount of time). Trajectories of phone events (and lat-lon of cell towers) between stay regions are extracted as her trips. Start and end times of a trip are estimated using linear extrapolation. Based on the start and end times, temporal patterns are extracted. Trips with sufficient number of intermediate points are mapped to transport network that consists of train lines, bus routes and expressways. We use Kernel density estimation to visualize the most common path for a given origin and destination. Based on this we create a round-the-clock visualization of mobility of people over the entire city separately for weekdays and weekends. At the end we show the validation results.

We are witnessing a new revolution in computing and communication involving symbiotic networks of people (social networks), intelligent devices, smart mobile computing, and communication devices that will form cyber-physical social systems. The emergence of intelligent devices with monitoring, sensing, and actuation capabilities referred to as Internet of Things and social networks have increased the popularity of novel social applications such as crowdsourcing and crowdsensing. The upsurge of such applications has fostered the need for scalable cost-efficient platforms that can enable distributed data analytics. In this paper, we propose CARDAP, a scalable, energy-efficient, generic and extensible component-based distributed data analytics platform for mobile crowdsensing (MCS) applications. CARDAP incorporates on-the-move activity recognition and a number of energy efficient data delivery strategies using real-time mobile data stream mining. We propose and develop theoretical cost models for typical crowdsensing application scenarios. Experimental evaluations of CARDAP using a proof-of-concept MCS scenario validate the theoretical cost model estimates and demonstrate the platform's ability to deliver significant benefits in energy, resource, and query processing efficiency.

One major problem of using location data collected from mobile cellular networks for mobility modelling is the oscillation phenomenon. An oscillation occurs when a mobile phone intermittently switches between cell towers instead of connecting to the nearest cell tower. For the purpose of mobility modeling, the location data needs to be cleansed to approximate the mobile device's actual location. However, this constitutes a challenge because the mobile device's true location is not known. In this paper, we study the oscillation resolution problem. We propose an algorithm framework called DECRE (Detect, Expand, Check, Remove) to detect and remove oscillation logs. To make informed decisions DECRE includes four steps: Detect, to identify log sequences that may contain oscillation using a few heuristics based on the concepts of stable period and moving at impossible speed, Expand, to look before and after suspicious records to gain more information, Check, to check whether a cell tower is observed repeatedly (which is a strong indication of oscillation), and Remove, resolving oscillation by selecting a cell tower to approximate the mobile device's actual location. Our experimental results on travel diaries show that our oscillation resolution approach is able to remove records that are far from mobile device's ground-truth locations, improve the quality of the location data, and performs better than an existing method. Our performance study on large scale cell tower data shows that the MapReduce implementation of our approach is able to process 1 Terabyte of cell tower data in five hours using a small cluster.

Most data stream classification techniques assume that the underlying feature space is static. However, in real-world applications the set of features and their relevance to the target concept may change over time. In addition, when the underlying concepts reappear, reusing previously learnt models can enhance the learning process in terms of accuracy and processing time at the expense of manageable memory consumption. In this paper, we propose mining recurring concepts in a dynamic feature space (MReC-DFS), a data stream classification system to address the challenges of learning recurring concepts in a dynamic feature space while simultaneously reducing the memory cost associated with storing past models. MReC-DFS is able to detect and adapt to concept changes using the performance of the learning process and contextual information. To handle recurring concepts, stored models are combined in a dynamically weighted ensemble. Incremental feature selection is performed to reduce the combined feature space. This contribution allows MReC-DFS to store only the features most relevant to the learnt concepts, which in turn increases the memory efficiency of the technique. In addition, an incremental feature selection method is proposed that dynamically determines the threshold between relevant and irrelevant features. Experimental results demonstrating the high accuracy of MReC-DFS compared with state-of-the-art techniques on a variety of real datasets are presented. The results also show the superior memory efficiency of MReC-DFS.

Mobile activity recognition focuses on inferring the current activities of a mobile user by leveraging the sensory data that is available on today's smart phones. The state of the art in mobile activity recognition uses traditional classification techniques. Thus, the learning process typically involves: i) collection of labelled sensory data that is transferred and collated in a centralised repository, ii) model building where the classification model is trained and tested using the collected data, iii) a model deployment stage where the learnt model is deployed on-board a mobile device for identifying activities based on new sensory data. In this paper, we demonstrate the Mobile Activity Recognition System (MARS) where for the first time the model is built and continuously updated on-board the mobile device itself using data stream mining. The advantages of the on-board approach are that it allows model personalisation and increased privacy as the data is not sent to any external site. Furthermore, when the user or its activity profile changes MARS enables quick model adaptation. One of the stand out features of MARS is that training/updating the model takes less than 30 seconds per activity. MARS has been implemented on the Android platform to demonstrate that it can achieve accurate mobile activity recognition. Moreover, we can show in practice that MARS quickly adapts to user profile changes while at the same time being scalable and efficient in terms of consumption of the device resources.

The problem of recurring concepts in data stream classification is a special case of concept drift where concepts may reappear. Although several existing methods are able to learn in the presence of concept drift, few consider contextual information

Advances in data mining, particularly in anytime anywhere data stream mining, make on-board data analysis possible in mobile devices with resource constraints. In this work, we propose a data stream mining service to support knowledge discovery in ubiquitous applications while addressing resource constraints on mobile devices. As the basis for our service we describe a general mechanism, which autonomously adapts the execution of the data stream mining process to each situation, using context and resource awareness. We describe the main components to achieve adaptability and propose a decision mechanism based on machine learning to support the configuration selection task, as we consider this to be a key element to achieve autonomy and adaptation of the mining service. We then present an instantiation of the proposed approach for the particular case of classification using the VFDT algorithm and analyze which factors influence it. Experimental results show how the adaptable data stream mining service improves resource consumption while increasing the quality of the anytime mining model.