Abstract Travel delays and bus overcrowding are some of the daily dissatisfactions of public transportation users. These problems may be caused by bus bunching, an event that occurs when two or more buses are running the same route together, i.e. out of schedule. Due to the stochastic nature of the traffic, a static schedule is not effective to avoid the occurrence of these events; thus, preventive actions are necessary to improve the reliability of the public transportation system. In this context, we propose a decision tree ensemble model to predict bus bunching. We use an ensemble of Random Forest, eXtreme Gradient Boosting and Categorical Boosting models applied to Global Positioning System, General Transit Feed Specification, weather and traffic situation data. The efficacy of the proposed model has been demonstrated using real data sets and has been compared with four baselines: Linear Regression, Logistic Regression, Support Vector Machine and Relevance Vector Machine. According to the results, the proposed model can achieve an efficacy between 74 and 80% and can be used to predict bus bunching in real time up to 10 stops before its occurrence.

Summary Several methods of entity resolution (ER) have been developed in academia and industry over the years, with the intention to identify duplicate entities (eg, records) in datasets. To evaluate the efficacy of such methods, it is necessary to compare their results with a ground‐truth, which consists of a document containing all known duplicate record pairs in a dataset. In general, the generation of ground‐truths for real datasets is performed manually from the inspection of all combinations of pairs of records in a dataset. This is subject to error and presents quadratic complexity, with respect to the size(s) of the dataset(s), requiring a long time to be performed. In this context, some works present (semi)automatic approaches for the generation of ground‐truths for the ER task. However, such approaches are either not applicable to several domains or still present a considerable manual effort. In this work, we propose GTGenERAL, a semiautomatic approach that combines results from multiple algorithms of ER together with active learning to generate accurate ground‐truths employing reduced manual effort. Experiments using real datasets show that, with great manual effort reduction, GTGenERAL is able to generate ground‐truths close to those generated by the state‐of‐the‐art approach.

The widespread use of information systems has become a valuable source of semi-structured data. In this context, Entity Resolution (ER) emerges as a fundamental task to integrate multiple knowledge bases or identify similarities between data items (i.e., entities). Since ER is an inherently quadratic task, blocking techniques are often used to improve efficiency. Beyond the challenges related to the data volume and heterogeneity, blocking techniques also face two other challenges: streaming data and incremental processing. To address these challenges, we propose PRIME, a novel incremental schema-agnostic blocking technique that utilizes parallelism to enhance blocking efficiency. The proposed technique deals with streaming and incremental data using a distributed computational infrastructure. To improve efficiency, the technique avoids unnecessary comparisons and applies a time window strategy to prevent excessive memory consumption.

The availability of historical data on the global positioning systems' trajectories of vehicles and passenger boarding information for public bus fleets of large municipalities has given researchers and practitioners the opportunity to explore new challenges regarding the analysis of public transportation systems. This paper performs one such analysis as a case study examining the margin of improvement that passengers of a 1.8M people Brazilian city have when choosing their daily bus trips. In doing so, we document a number of not readily apparent challenges that must be overcome to leverage public transportation big data to policymakers, transportation systems operators, and citizens. Solutions are devised to each of these challenges and demonstrated on the analysis of the aforementioned 1.8M people city.

As cities are becoming green and smart, public information systems are being revamped to adopt digital technologies. There are several sources (official or not) that can provide information related to a city. The availability of multiple sources enables the design of advanced analyses for offering valuable services to both citizens and municipalities. However, such analyses would fail if the considered data were affected by errors and uncertainties: Data Quality is one of the main requirements for the successful exploitation of the available information. This paper highlights the importance of the Data Quality evaluation in the context of geographical data sources. Moreover, we describe how the Entity Matching task can provide additional information to refine the quality assessment and, consequently, obtain a better evaluation of the reliability data sources. Data gathered from the public transportation and urban areas of Curitiba, Brazil, are used to show the strengths and effectiveness of the presented approach.

Blocking techniques are widely applied in Entity Resolution (ER) approaches as preprocessing step in order to avoid the quadratic cost of the ER task. In this context, heterogeneous data and Big Data emerges as the major challenges that are faced by blocking techniques. In this sense, we propose the novel approach Spark-based Streamlined Metablocking (SS-Metablocking). Moreover, this work proposes the Cardinality-based load balancing technique to be applied in SS-Metablocking in order to improve its efficiency. To improve the effectiveness of the SS-Metablocking, the GWNP pruning algorithm is proposed in this work. Based on the experimental results, we can highlight that the proposed approach presents better results regarding efficiency and effectiveness than the state-of-the-art approach.

The effectiveness and scalability of MapReduce-based implementations for data-intensive tasks depends on the data assignment made from map to reduce tasks. The robustness of this assignment strategy is crucial to achieve skewed data handling and balanced workload distribution among all reduce tasks. For the entity matching problem in the Big Data context, we propose BlockSlicer, a MapReduce-based approach that supports blocking techniques to reduce the entity matching search space. The approach utilizes a preprocessing MapReduce job to analyze the data distribution and provides an improved load balancing by applying an efficient block slice strategy as well as a well-known optimization algorithm to assign the generated match tasks. We evaluate the approach against an existing one that addresses the same problem on a real cloud infrastructure. The results show that our approach increases significantly the performance of distributed entity matching task by reducing the amount of data generated from the map phase and diminishing the overall execution time.