Continuous data streams, generated by modern sensed cities, open many opportunities and perspectives in terms of developing new innovative services. To exploit this potential, flexible and scalable platforms are needed to ease the design, development, deployment, and operations of new city services. In recent years, several problem-specific platforms have been proposed in different application domains; however, to boost the evolution of smart cities, we claim the need for city-oriented platforms that can be easily customized to address different day-to-day life challenging problems. In this paper, we present the main architectural challenges and solutions proposed for the design of a novel open-source platform (named PROMENADE) characterized by: i ) a data-driven graph-based modeling support to ensure high generality for addressing disparate problems related to the networked nature of many city infrastructures and systems, ii ) the dynamic nature of the graph entities updated in real-time from different sources ( e.g., IoT/Edge networks, data providers, etc.), and iii ) high efficiency, scalability and flexibility to easily support new city services. The platform is designed around a general-purpose core that provides a set of built-in standard features such as data ingestion , storage, processing, and visualization exposed as a collection of containerized microservices . A specialization of the platform has been developed for road networks monitoring. It has been deployed in OpenShift/Kubernetes and tested using realistic datasets collected from the city of Lyon, France. The analysis addresses an important problem of big data processing pipelines: the synchronization between data ingestion and processing in order to produce an accurate result in useful time. To this end, we study different approaches for synchronization and show how the end-to-end latency is kept under control by leveraging the scalability of the platform.

Real-time anomaly detection in urban areas from massive data is a recent research field with challenging requirements.This paper presents a lightweight framework for real-time anomaly detection in multivariate time-series extracted from large-scale Mobile-phone Network Data (MND).Our solution relies on unsupervised machine learning applied to MND collected at individual antennas of a nation-wide French mobile phone network operator.The proposed framework is based on a two-step approach: (i) the offline stage aims at assessing the typical behaviour of the antennas; (ii) the online stage performs real-time comparison of incoming data with respect to the detected typical behaviour.Results related to a real case-study of terrorist attack in the city of Lyon show that our framework can successfully detect an emergency event almost instantaneously and locate the anomalous area with high precision.

Personal trajectory data are becoming more and more accessible and have a high value in transport planning and mobility characterisation, at the cost of a risk for user's privacy.Addressing this risk is usually computationally expensive and can lead to losing most of the data utility.We explore a new, light-weight approach to Origin/Destinationmatrix anonymization that is easily scalable.We apply it to trip records from New York City Taxi and Limousine Commission (TLC) to illustrate how it can combine foolproof anonymity with a good spatial precision for a reasonable computational cost.

Nowadays a large amount of data is originated by complex systems, such as social networks, transportation systems, computer and service networks. These systems can be effectively modeled through graphs and studied by exploiting graph metrics, such as Betweenness Centrality (BC), a popular metric to analyze node centrality. In spite of its great potential, this metric requires long computation time, especially for large graphs. In this paper, we present a novel very fast algorithm to compute exact BC of undirected, scale-free graphs. The algorithm is based on clustering and exploits structural properties of graphs to find classes of equivalent nodes. By selecting one representative node for each class, we are able to calculate BC by significantly reducing the number of single-source shortest path explorations adopted by the Brandes' algorithm. The experimental evaluation of both sequential and map-reduce parallel versions reveals that our solution largely outperforms Brandes and recent heuristics, especially for large graphs while preserving good scalability.

The ongoing digitization of cities, enabled by the diffusion of interconnected sensors and devices, makes it possible to continuously collect and analyze huge streams of data at extremely large spatio-temporal scales and fine resolutions. These data can be used to monitor, detect and anticipate different kinds of infrastructure vulnerabilities and anomalies, as well as to implement more personalized services that could improve citizens' life. In this new context, full of opportunities, it is difficult to foresee and develop, in advance, the set of applications and services that can be potentially useful for administrators and citizens to solve the manifold compelling needs a city may have to face. Novel ICT paradigms and technologies can help designing agile, general-purpose smart city platforms aimed at supporting the collection and treatment of large-scale, multi-source (streams of) data and the development of novel applications that could fulfill diverse functional requirements under strict non-functional constraints. This paper presents the reference architecture, a prototype implementation and a city-scale case-study evaluation of PROMENADE, a platform that exploits IoT/Fog/Cloud paradigms, microservices and DevOps infrastructures to guarantee continuous development of robust and reliable applications for real-time monitoring and analysis of traffic data generated by IoT devices in large smart cities. The prototype has been evaluated in a case study concerning the quasi real-time detection of road networks vulnerabilities via centrality measures from on-line traffic conditions, emulated from off-line real datasets available for the city of Lyon, France.

Human mobility analysis is a multidisciplinary research subject that has attracted a growing interest over the last decade. A substantial amount of such recent studies is driven by the availability of original sources of real-world information about individual movement patterns. An important task in the analysis of mobility data is reliably distinguishing between the stop locations and movement phases that compose the trajectories of the monitored subjects. The problem is especially challenging when mobility is inferred from mobile phone location data: here, oscillations in the association of mobile devices to base stations lead to apparent user mobility even in absence of actual movement. In this paper, we leverage a unique dataset of spatiotemporal individual trajectories that allows capturing both the user and network operator perspectives in mobile phone location data, and investigate the oscillation phenomenon. We present probabilistic and machine learning approaches for detecting oscillations in mobile phone location data, and a filtering technique for removing those. Our analyses and comparison with state-of-the-art approaches demonstrate the superiority of our solution, both in terms of removed oscillations and of error with respect to ground-truth trajectories.

Computation of node betweenness centrality (BC) of weighted and directed graphs is a time-consuming task that could limit the application of such a metric for monitoring large, dynamic networks. As widely demonstrated in previous work, approximated approaches represent a solution to reduce computation time when ranking nodes according to their BC values is sufficient with respect to knowing their exact BC values. According to this observation, we have proposed a fast algorithm for computing approximated BC values for large weighted and directed graphs. It is based on the identification of pivot nodes that equally contribute to BC values of the other nodes of the network discovered via a cluster-based approach.In this paper, we focus on the performance and scalability analysis of the proposed algorithm in order to characterize its behavior with different sets of computing resources and to identify room for further improvements. To this end, we exploit a real dataset related to a transportation network. The results show that the proposed algorithm exhibits significantly lower execution times if compared with the Brandes's solution for computing exact BC values, especially when the number of available computing resources is limited. However, the speedup is not negligible even when the number of resources grows, where improvements are possible, as shown by our analysis.

The ability to detect critical spots in transportation networks is fundamental to improve traffic operations and road-network resilience in smart cities. Real-time monitoring of these networks, especially in very large metropolitan areas, is a compelling challenge due to the complexity of computing robustness metrics. This paper presents a framework for identifying vulnerabilities in very-large road networks. The framework adopts graph-based modeling of road networks and exploits big-data techniques and technologies for processing such large and complex graphs. First, we use the framework to prove the existence of a significant correlation between global efficiency and betweenness centrality. Then, we focus on an efficient algorithm, integrated in the framework, to rank the nodes according to this metric for finding potential vulnerabilities of a road network. To keep computation time under a "quasi" real-time threshold, a fast, requirement-driven, approximated strategy for computing betweenness centrality is adopted. The evaluation shows that the algorithm, integrated in the framework, exhibits a very good approximation for the most critical nodes, thus being well-suited for on-line operational monitoring.

Betweenness centrality is a metric widely used in several domains (social, biological, transportation, computer) to identify critical nodes of networks. Its exact computation is very demanding, with an O(nm) time complexity for unweighted graphs (where n is the number of nodes and m is the number of edges). Such complexity becomes an obstacle to the adoption of betweenness centrality for continuous monitoring of critical nodes in very large networks. Several solutions have been proposed to reduce computation time, mainly via parallelism, approximation or incremental recalculation. In this paper, we propose an algorithm for computing approximated values of betweenness that allows for tuning its performance on the basis of a tolerable error. The algorithm aims at reducing the number of single-source shortest-paths explorations via a pivot-based technique that exploits topological properties of graphs and clustering. It is evaluated by identifying the vulnerabilities (critical nodes) of a real-world, very-large road network. The evaluation shows that the approximation error does not significantly affect the most critical nodes, thus making the algorithm well-suited for on-line operational monitoring of road networks.

Mobile traffic data collected by network operators is a rich source of information about human habits, and its analysis provides insights relevant to many fields, including urbanism, transportation, sociology and networking. In this paper, we present an original approach to infer both spatial and temporal structures hidden in the mobile demand, via a first-time tailoring of Exploratory Factor Analysis (EFA) techniques to the context of mobile traffic datasets. Casting our approach to the time or space dimensions of such datasets allows solving different problems in mobile traffic analysis, i.e., network activity profiling and land use detection, respectively. Tests with real-world mobile traffic datasets show that, in both its variants above, the proposed approach (i) yields results whose quality matches or exceeds that of state-of-the-art solutions, and (ii) provides additional joint spatiotemporal knowledge that is critical to result interpretation.

Urban landscapes present a variety of socio-topological environments that are associated to diverse human activities. As the latter affect the way individuals connect with each other, a bound exists between the urban tissue and the mobile communication demand. In this paper, we investigate the heterogeneous patterns emerging in the mobile communication activity recorded within metropolitan regions. To that end, we introduce an original technique to identify classes of mobile traffic signatures that are distinctive of different urban fabrics. Our proposed technique outperforms previous approaches when confronted to ground-truth information, and allows characterizing the mobile demand in greater detail than that attained in the literature to date. We apply our technique to extensive real-world data collected by major mobile operators in 10 cities. Results unveil the diversity of baseline communication activities across countries, but also provide evidence of the existence of a number of mobile traffic signatures that are common to all studied areas and specific to particular land uses.

In the next few years, mobile networks will undergo significant evolutions in order to accommodate the ever-growing load generated by increasingly pervasive smartphones and connected objects. Among those evolutions, cognitive networking upholds a more dynamic management of network resources that adapts to the significant spatiotemporal fluctuations of the mobile demand. Cognitive networking techniques root in the capability of mining large amounts of mobile traffic data collected in the network, so as to understand the current resource utilization in an automated manner. In this paper, we take a first step towards cellular cognitive networks by proposing a framework that analyzes mobile operator data, builds profiles of the typical demand, and identifies unusual situations in network-wide usages. We evaluate our framework on two real-world mobile traffic datasets, and show how it extracts from these a limited number of meaningful mobile demand profiles. In addition, the proposed framework singles out a large number of outlying behaviors in both case studies, which are mapped to social events or technical issues in the network.

Mobile traffic data has been recently used to characterize the urban environment in terms of urban fabric profiles. While showing promising results, the existing urban fabric detection solutions are built without a clear understanding of the detection process chain. In this paper, we distinguish and analyze the different steps common to all urban profiling techniques. By evaluating the impact of each step of the process, we are able to propose a new solution that outperforms the state of the art techniques. Our approach uses the weekly periodicity of human activities, as well as a median-based filtering technique, resulting in a better clustering in terms of both coverage and entropy, as shown by results obtained on two large scale mobile traffic datasets covering the urban areas of Milan and Turin, in Italy.

Unstructured peer-to-peer (P2P) architectures offer several benefits to implement semantic discovery and composition in future-generation service registries. However, their success strongly depends on the adoption of efficient techniques for disseminating semantic queries over the network. Gossip strategies significantly reduce the amount of messages with respect to flooding, but they need a predefined tuning of the effectual fanout to achieve good performance. In this paper, we compare typical gossip strategies with our proposal, which is able to dynamically exploit network knowledge to fulfil a selective choice of propagation paths in order to ensure high recall and further reduce the number of messages exchanged. We perform the comparison in a simulated environment to observe resolution time, recall and message overhead on large-size and evolving networks while searching for service compositions. We have adopted Bernoulli, Random Geometric and Scale-Free graphs to model different network topologies. The experimental results show that our approach is able to adapt to network changes and preserve high levels of recall. In particular, it reduces message overhead, with respect to both optimized flooding and the analysed gossip-based strategies, or improves the recall, whereas resolution time remains almost unchanged.

In this paper, we propose an efficient technique for improving the performance of automatic and cooperative compositions in P2P unstructured networks during service discovery. Since the adoption of flooding to exchange queries and partial solutions among the peers of unstructured networks generates a huge amount of messages, the technique exploits a probabilistic forwarding algorithm that uses different sources of knowledge, such as network density and service grouping, to reduce the amount of messages exchanged. The technique, analyzed in several network configurations by using a simulator to observe resolution time, recall and message overhead, has shown good performances especially in dense and large-scale service networks.

Event-driven programming is progressively replacing the call-stack model to improve flexibility, efficiency and scalability in SOA. Enterprise applications often deal with large messages attached to asynchronous events. This could reduce the benefits provided by event-driven programming since the need for having every information propagated as event is counterbalanced by wasting resources when large messages are entirely propagated to destinations that do not use all of them. In this paper, we propose the adoption of the D-WSLink framework for improving data transfers by using a composite and extensible declarative mechanism to inject the desired message transfer strategies into the underlying middleware. At the current stage, we focus mainly on (conditional) lazy transfer mechanisms even though the framework is able to support also smarter strategies. In particular, we compare, through an experimental analysis, our system with Apache Camel in delivering events with large attachments. The results show that the proposed approach is effective not only for programming but also at performance level.