We describe a production Twitter system for generating relevant, personalized, and timely recommendations based on observing the temporally-correlated actions of each user's followings. The system currently serves millions of recommendations daily to tens of millions of mobile users. The approach can be viewed as a specific instance of the novel problem of online motif detection in large dynamic graphs. Our current solution partitions the graph across a number of machines, and with the construction of appropriate data structures, motif detection can be translated into the lookup and intersection of adjacency lists in each partition. We conclude by discussing a generalization of the problem that perhaps represents a new class of data management systems.

Micro-blogging services have become indispensable communication tools for online users for disseminating breaking news, eyewitness accounts, individual expression, and protest groups. Recently, Twitter, along with other online social networking services such as Foursquare, Gowalla, Facebook and Yelp, have started supporting location services in their messages, either explicitly, by letting users choose their places, or implicitly, by enabling geo-tagging, which is to associate messages with latitudes and longitudes. This functionality allows researchers to address an exciting set of questions: 1) How is information created and shared across geographical locations, 2) How do spatial and linguistic characteristics of people vary across regions, and 3) How to model human mobility. Although many attempts have been made for tackling these problems, previous methods are either complicated to be implemented or oversimplified that cannot yield reasonable performance. It is a challenge task to discover topics and identify users' interests from these geo-tagged messages due to the sheer amount of data and diversity of language variations used on these location sharing services. In this paper we focus on Twitter and present an algorithm by modeling diversity in tweets based on topical diversity, geographical diversity, and an interest distribution of the user. Furthermore, we take the Markovian nature of a user's location into account. Our model exploits sparse factorial coding of the attributes, thus allowing us to deal with a large and diverse set of covariates efficiently. Our approach is vital for applications such as user profiling, content recommendation and topic tracking. We show high accuracy in location estimation based on our model. Moreover, the algorithm identifies interesting topics based on location and language.

In recent years social media have become indispensable tools for information dissemination, operating in tandem with traditional media outlets such as newspapers, and it has become critical to understand the interaction between the new and old sources of news. Although social media as well as traditional media have attracted attention from several research communities, most of the prior work has been limited to a single medium. In addition temporal analysis of these sources can provide an understanding of how information spreads and evolves. Modeling temporal dynamics while considering multiple sources is a challenging research problem. In this paper we address the problem of modeling text streams from two news sources - Twitter and Yahoo! News. Our analysis addresses both their individual properties (including temporal dynamics) and their inter-relationships. This work extends standard topic models by allowing each text stream to have both local topics and shared topics. For temporal modeling we associate each topic with a time-dependent function that characterizes its popularity over time. By integrating the two models, we effectively model the temporal dynamics of multiple correlated text streams in a unified framework. We evaluate our model on a large-scale dataset, consisting of text streams from both Twitter and news feeds from Yahoo! News. Besides overcoming the limitations of existing models, we show that our work achieves better perplexity on unseen data and identifies more coherent topics. We also provide analysis of finding real-world events from the topics obtained by our model.

Traditional web search engines find it challenging to achieve good search quality for recency-sensitive queries, as they are prone to delays in discovering, indexing and ranking new web pages. In this paper we introduce PreGen, an adaptive preview generation system, which is run as part of a web search engine to improve search result quality for recency-sensitive queries. PreGen uses a machine learning algorithm to classify and select live web feeds, and generates "previews" of new web pages based on the link descriptions available in these feeds. The search engine can then index and present relevant page previews as part of its search results before the pages are fetched from the web, thereby reducing end-to-end delays. Our experiments show that PreGen improves the search relevance of a state-of-the-art search engine for recency-sensitive queries by 3% and reduces the average latencies of affected documents by 50%.

We propose a middleware called BuddyShare to automatically form an overlay group of nearby friends' mobile phones to collaboratively download data by sharing mobile internet. This system is hypothetical in nature and only work on certain assumptions such as: 1) frequent availability of friends' phone nearby, 2) Sufficient social trust among physically close users to share internet and 3) sufficient social networking information available in phones. In order to validate these hypotheses, we collected human centric dataset of cellular phone users of university environment to study the user behavior. In this paper, we present certain social and proximity behaviors of these users that validate these hypotheses and show the practical feasibility of a BuddyShare system. We also study the usefulness of BuddyShare by virtually leveraging it on this user network, which concludes around three times scaling in download rate on average.

With the ever-growing competition in telecommunications markets, operators have to increasingly rely on business intelligence to offer the right incentives to their customers. Existing approaches for telecom business intelligence have almost solely focused on the individual behavior of customers. In this paper, we use the call detail records of a mobile operator to construct call graphs, that is, graphs induced by people calling each other. We determine the structural properties of these graphs and also introduce the Treasure-Hunt model to describe the shape of mobile call graphs. Moreover, we determine how the structure of these call graphs evolve over time. Finally, since short messaging service (SMS) is becoming a preferred mode of communication among many sections of the society, we study the properties of the SMS graph. Our analysis indicates several interesting similarities and differences between the SMS graph and the corresponding call graph. We believe that our analysis techniques can allow telecom operators to better understand the social behavior of their customers and potentially provide major insights for designing effective incentives.

With ever growing competition in telecommunications markets, operators have to increasingly rely on business intelligence to offer the right incentives to their customers. Toward this end, existing approaches have almost solely focussed on the individual behaviour of customers. Call graphs, that is, graphs induced by people calling each other, can allow telecom operators to better understand the interaction behaviour of their customers, and potentially provide major insights for designing effective incentives.In this paper, we use the Call Detail Records of a mobile operator from four geographically disparate regions to construct call graphs, and analyse their structural properties. Our findings provide business insights and help devise strategies for Mobile Telecom operators. Another goal of this paper is to identify the shape of such graphs. In order to do so, we extend the well-known reachability analysis approach with some of our own techniques to reveal the shape of such massive graphs. Based on our analysis, we introduce the Treasure-Hunt model to describe the shape of mobile call graphs. The proposed techniques are general enough for analysing any large graph. Finally, how well the proposed model captures the shape of other mobile call graphs needs to be the subject of future studies.

A telecom operator ("service provider",SP) offers various services to subscribed customers by partnering with various third party providers ("content provider",CP). The SP acts as a liaison between subscribers and partners. One of the main functions of the SP, therefore, is to match the "demand" of the subscribers with the "supply" of the CPs. Such a matching is a prerequisite for efficient service selection while ensuring customer satisfaction, and is useful for optimisation, such as resource allocation and load balancing. The "demand" requirements and "supply" guarantees can be concretized using Service Level Agreements (SLAs). SLAs can be expressed formally using standards such as WSLA or WS-Agreement. We present a system that automates the task of finding a matching between these two sets, subscriber-SP and SP-CP, of SLAs. First, the SLAs are normalised to a common denominator, then composed if required, and finally the matching engine computes and outputs the map. The matching algorithm, which is of central importance, compares logical expressions involving predicates. The logical expressions are first converted into CNFform, and instead of naive O(m × n) comparisons, we develop a more efficient approach to solve the problem. As a proof-of-concept, we have implemented a prototype as an Eclipse plugin.