CDR (Call Detail Records) data is more easily available than other network related data (such as GPS data) as most telecommunications service providers (TSPs) maintain such data. By analyzing it one can find mobility patterns of most of the population thus leading to efficient urban planning, disease and traffic control, etc. But its granularity is low as the latitude and longitude (lat-lon) of a cell tower is used as the current location of all mobile phones that are connected to the cell tower at that time. Granularity can range between 10s of metres to several kms depending on population density of a locality. This is one reason why, although there are many existing systems on visualizing mobility of people based on GPS data, there is hardly any existing system for CDR. We develop a Mobility Visualization System (MoVis) for visualizing mobility of people from their CDR records. First of all, given the CDR records of a user, we determine her stay regions (places where she stays for a significant amount of time). Trajectories of phone events (and lat-lon of cell towers) between stay regions are extracted as her trips. Start and end times of a trip are estimated using linear extrapolation. Based on the start and end times, temporal patterns are extracted. Trips with sufficient number of intermediate points are mapped to transport network that consists of train lines, bus routes and expressways. We use Kernel density estimation to visualize the most common path for a given origin and destination. Based on this we create a round-the-clock visualization of mobility of people over the entire city separately for weekdays and weekends. At the end we show the validation results.

In this paper we present methods for mapping trajectories of cell tower latitude-longitudes in mobile phone data to transport network. Compared to other forms of mobile phone data such as GPS traces, this form is the most popular and inexpensive but the least accurate and highly irregular. Unlike some earlier works on this type of data where only long trips are studied, our work is based on everyday transportation inside a city. The problems due to inaccuracy in determining position of a user and irregularity of phone events are felt even more intensely for such small distances. Our approach in processing this data takes into account its repetitive nature, i.e., A typical user makes the same trips every week. Moreover, many users follow the same transport route to go from one place to another. First of all, we defined stay region taking into consideration irregularity of phone events. Then we extracted trajectories between each pair of stay regions as trips. Then, we map these trips to existing transportation network consisting of bus services, train lines, expressways, and their combinations such as bus-train-bus. For a given origin and destination, we create a heat map that shows the most commonly used route. A round-the-clock heat map, both for weekdays and weekends, shows people's mobility at different times of a week. We validate our findings by both direct (sample survey) and indirect (comparison with available statistics) methods.

Knowledge about population distribution of planning areas helps in making urban development decisions. Two important criteria are: "where do people live?" and "where do they work?" In this paper we propose methods to find home and workplaces from mobile network data. Home and work places are essential for discovery of mobility profiles of users. Validation of home and workplace prediction is not straight forward. We validate our methods using correlation with external data. Validation results show that even though a single cellular provider has only a portion of the entire population as its users, distribution of home and work places predicted using its mobile network data match that of government statistics. On the basis of this matching, we can have faith in distributions of more difficult statistics extracted from mobile network data which are difficult to obtain from external sources. We implemented an interactive system to show various distributions such as people living and working in different planning areas, and people working in different job sectors such as manufacturing. Interesting relationships are found by calculating joint distributions, e.g., Where do people, living in a planning area, work, and vice versa. Planning areas are ranked by the average distance traveled from home to work. Another interesting fact we extract is balance. Balance of a planning area is high if people live and work there, it is low if people living in a planning area work in other planning areas. We extend these statistics to regions which consist of many planning areas. The goal of this interactive system is to understand location semantics and mobility of users to aid in making urban development decisions. A video recording with subtitles is uploaded in http://www.youtube.com/watch?v=mo-7-DsCymw.

We present methods to predict and validate home and work places of anonymized users using their mobile network data. Knowledge of home and work place of a user is essential in order to find his (and overall population) mobility profiles. There are many methods that predict home and work places using GPS data. But unlike GPS data, mobile network data using GSM do not provide the exact location of a phone event. We use a novel criterion that combines an extracted feature from mobile data (i.e., Inactivity - no phone event for a given period of time) with open source data about location category % (i.e., Streetdirectory.com) to predict home location. Results show that the new criterion gives better prediction accuracy than inactivity alone. We predict work place using the idea that one goes to her work place on most of the weekdays but rarely on weekends. We validate our methods by comparing against the ground truth obtained from open source data. Validation results show that our proposed methods are about 25% more accurate than existing methods both for home and work place predictions.

One major problem of using location data collected from mobile cellular networks for mobility modelling is the oscillation phenomenon. An oscillation occurs when a mobile phone intermittently switches between cell towers instead of connecting to the nearest cell tower. For the purpose of mobility modeling, the location data needs to be cleansed to approximate the mobile device's actual location. However, this constitutes a challenge because the mobile device's true location is not known. In this paper, we study the oscillation resolution problem. We propose an algorithm framework called DECRE (Detect, Expand, Check, Remove) to detect and remove oscillation logs. To make informed decisions DECRE includes four steps: Detect, to identify log sequences that may contain oscillation using a few heuristics based on the concepts of stable period and moving at impossible speed, Expand, to look before and after suspicious records to gain more information, Check, to check whether a cell tower is observed repeatedly (which is a strong indication of oscillation), and Remove, resolving oscillation by selecting a cell tower to approximate the mobile device's actual location. Our experimental results on travel diaries show that our oscillation resolution approach is able to remove records that are far from mobile device's ground-truth locations, improve the quality of the location data, and performs better than an existing method. Our performance study on large scale cell tower data shows that the MapReduce implementation of our approach is able to process 1 Terabyte of cell tower data in five hours using a small cluster.

Actionable 3D subspace clustering from real-world continuous-valued 3D (i.e., object-attribute-context) data promises tangible benefits such as discovery of biologically significant protein residues and profitable stocks, but existing algorithms are inadequate in solving this clustering problem; most of them are not actionable (ability to suggest profitable or beneficial actions to users), do not allow incorporation of domain knowledge, and are parameter sensitive, i.e., the wrong threshold setting reduces the cluster quality. Moreover, its 3D structure complicates this clustering problem. We propose a centroid-based actionable 3D subspace clustering framework, named CATSeeker, which allows incorporation of domain knowledge, and achieves parameter insensitivity and excellent performance through a unique combination of singular value decomposition, numerical optimization, and 3D frequent itemset mining. Experimental results on synthetic, protein structural, and financial data show that CATSeeker significantly outperforms all the competing methods in terms of efficiency, parameter insensitivity, and cluster usefulness.

The presence of noise or errors in the stated feature values of biomedical data can lead to incorrect prediction. We introduce a Bayesian Network-based Noise Correction framework named BN-NC. After data preprocessing, a Bayesian Network (BN) is learned to capture the feature dependencies. Using the BN to predict each feature in turn, BN-NC estimates a feature's error rate as the deviation between its predicted and stated values in the training data, and allocates the appropriate uncertainty to its subsequent findings during prediction. BN-NC automatically generates a probabilistic rule to explain BN prediction on the class variable using the feature values in its Markov blanket, and this is reapplied as necessary to explain the noise correction on those features. Using three real-life benchmark biomedical data sets (on HIV-1 drug resistance prediction and leukemia subtype classification), we demonstrate that BN-NC (1) accurately detects the errors in biomedical feature values, (2) automatically corrects for the errors to maintain higher prediction accuracy over competing methods including Decision Trees, Naive Bayes and Support Vector Machines, and (3) generates probabilistic rules that concisely explain the prediction and noise correction decisions. In addition to achieving more robust biomedical prediction in the presence of feature noise, by highlighting erroneous features and explaining their corrections, BN-NC provides medical researchers with high utility insights to biomedical data not found in other methods.

In content-based image retrieval, the ldquosemantic gaprdquo between visual image features and user semantics makes it hard to predict abstract image categories from low-level features. We present a hybrid system that integrates global features (G-features) and region features (R-features) for predicting image semantics. As an intermediary between image features and categories, we introduce the notion of mid-level concepts, which enables us to predict an image's category in three steps. First, a G-prediction system uses G-features to predict the probability of each category for an image. Simultaneously, a R-prediction system analyzes R-features to identify the probabilities of mid-level concepts in that image. Finally, our hybrid H-prediction system based on a Bayesian network reconciles the predictions from both R-prediction and G-prediction to produce the final classifications. Results of experimental validations show that this hybrid system outperforms both G-prediction and R-prediction significantly.

Acquisition of context poses unique challenges to mobile context-aware recommender systems. The limited resources in these systems make minimizing their context acquisition a practical need, and the uncertainty in the mobile environment makes missing and erroneous context inputs a major concern. In this paper, we propose an approach based on Bayesian networks (BNs) for building recommender systems that minimize context acquisition. Our learning approach iteratively trims the BN-based context model until it contains only the minimal set of context parameters that are important to a user. In addition, we show that a two-tiered context model can effectively capture the causal dependencies among context parameters, enabling a recommender system to compensate for missing and erroneous context inputs. We have validated our proposed techniques on a restaurant recommendation data set and a Web page recommendation data set. In both benchmark problems, the minimal sets of context can be reliably discovered for the specific users. Furthermore, the learned Bayesian network consistently outperforms the J4.8 decision tree in overcoming both missing and erroneous context inputs to generate significantly more accurate predictions.

Mobile context-aware recommender systems face unique challenges in acquiring context. Resource limitations make minimizing context acquisition a practical need, while the uncertainty inherent to the mobile environment makes missing context values a major concern. This paper introduces a scalable mechanism based on Bayesian network learning in a tiered context model to overcome both of these challenges. Extensive experiments on a restaurant recommender system showed that our mechanism can accurately discover causal dependencies among context, thereby enabling the effective identification of the minimal set of important context for a specific user and task, as well as providing highly accurate recommendations even when context values are missing.

Traditional approaches to recommender systems have not taken into account situational information when making recommendations, and this seriously limits the relevance of the results. This paper advocates context-awareness as a promising approach to enhance the performance of recommenders, and introduces a mechanism to realize this approach. We present a framework that separates the contextual concerns from the actual recommendation module, so that contexts can be readily shared across applications. More importantly, we devise a learning algorithm to dynamically identify the optimal set of contexts for a specific recommendation task and user. An extensive series of experiments has validated that our system is indeed able to learn both quickly and accurately.