Text representation models are the fundamental basis for information retrieval and text mining tasks. Although different text models have been proposed, they typically target specific task aspects in isolation, such as time efficiency, accuracy, or applicability for different scenarios. Here we present Bag of Textual Graphs (BoTG), a general text representation model that addresses these three requirements at the same time. The proposed textual representation is based on a graph‐based scheme that encodes term proximity and term ordering, and represents text documents into an efficient vector space that addresses all these aspects as well as provides discriminative textual patterns. Extensive experiments are conducted in two experimental scenarios—classification and retrieval—considering multiple well‐known text collections. We also compare our model against several methods from the literature. Experimental results demonstrate that our model is generic enough to handle different tasks and collections. It is also more efficient than the widely used state‐of‐the‐art methods in textual classification and retrieval tasks, with a competitive effectiveness, sometimes with gains by large margins.

The web is a large repository of entity-pages. An entity-page is a page that publishes data representing an entity of a particular type, for example, a page that describes a driver on a website about a car racing championship. The attribute values published in the entity-pages can be used for many data-driven companies, such as insurers, retailers, and search engines. In this article, we define a novel method, called SSUP , which discovers the entity-pages on the websites. The novelty of our method is that it combines URL and HTML features in a way that allows the URL terms to have different weights depending on their capacity to distinguish entity-pages from other pages, and thus the efficacy of the entity-page discovery task is increased. SSUP determines the similarity thresholds on each website without human intervention. We carried out experiments on a dataset with different real-world websites and a wide range of entity types. SSUP achieved a 95% rate of precision and 85% recall rate. Our method was compared with two state-of-the-art methods and outperformed them with a precision gain between 51% and 66%.

Excessive alcohol consumption is a worldwide problem, and social networks such as Twitter can provide valuable data that help understanding factors related to alcoholism, particularly among youngsters. The identification of drunk tweets (i.e. posted under the influence of alcohol) is complex because tweets are short, sparse and written with diverse and internet specific vocabulary, possibly with errors due to alcohol influence. In this paper, we propose an enriching framework that integrates conceptual and semantic features that expand and generalize the vocabulary, providing context to tweet terms. It also handles misspellings and the selection of discriminative features resulting from contextual enrichment. We outperformed the baseline, achieving improvements of 13.79 percentage points in recall, with no significant harm to precision. We illustrate the value of drunk tweets classification by developing an exploratory analysis that reveals drunk tweeters demographics and tweet properties.

The challenges in Big Data start during the data acquisition, where it is necessary to transform non-structured data into a structured format. The Big Data Era brought a new challenge to data acquisition: the need for eliminating user intervention. In this context, the domain-centric data extraction (DCDE) methods arose through replacing user intervention with content redundancy. The DCDE methods extract the attribute values of entities of the web that are restricted to a specific application domain. A research gap was identified from various analyzed methods, i.e., the DCDE methods are not effective in extracting attribute values that are differently presented within the same website. We have responded to this research gap by proposing the R-Extractor method, which extends a state-of-the-art DCDE method by adding a reinforcement stage for the treatment of attribute values that are differently presented within the same website. The R-Extractor method re-analyzes the extraction rules to identify those that can be combined to extract the values of a given attribute from all the pages that describe entities on a website. This identification is based on a novel score function that takes into account different features of the extraction rules. We carried out experiments on a dataset with more than 50k web pages from different real-world websites of a wide range of application domains. The R-Extractor method reached 98% of precision. Our method was compared with two baselines (an XPath-based method and a tree-based method) and outperformed them with an increase in precision up to 14%.

With the emergence of smartphones and location-based social networks, a large amount of user-generated data has become available to better understand city dynamics and help urban planning. While most of the related works choose to focus on specific dimensions of the data, the proposed models aim to benefit from the extent of the information existing in social platforms. Thus, this paper explores the full potential of social media data and proposes novel clustering models for retrieving information about city dynamics and urban characterization by incorporating new dimensions to state of the art algorithms. Aspects such as venue rating, entropy and popularity may lead to new and more complete understanding of activities and trends in a city. Preliminary experiments show that it is possible to aggregate a large diversity of information from different social networks, and generate different and complementary visualizations of the city. Moreover, by applying these methods to urban environments, governments and citizens can better understand and build better sustainable cities together.

In recent years several data classification techniques have been proposed.However, it is not a trivial task to choose the most appropriate classifier for deal with a particular problem and set it up properly.In addition, there is no optimal algorithm to solve all prediction problems.In order to improve the result of the classification process, the stacking strategy combines the knowledge acquired by individual learning algorithms aiming to discover new patterns not yet identified.Stacking combines the outputs of base classifiers, induced by several learning algorithms using the same dataset, by means of a meta-classifier.The main goal of this paper is to evaluate the effects of classifier diversity on the accuracy of stacking.We have performed a lot of experiments which results show the impact of multiple diversity measures on the gain of stacking, considering many real datasets extracted from UCI machine learning repository and three synthetic twodimensional datasets.The results revealed connections between some measures and the gain of stacking, but they imply a weak or moderate relationship that suggest predicting the improvement on the best base classifier accuracy using diversity measures is inappropriate.

Record deduplication aims at identifying entities that are potentially the same in a data repository. A set of pairs that is manually labeled is generally used to tune the deduplication process, as each dataset has a particular dirtiness pattern. However, producing an informative set of pairs is a very costly task, especially in very large datasets (even for expert users). We propose a new sampling strategy that is able to select a very small and informative set of pairs from large datasets. Our results show that our approach reduces user effort substantially while achieving a competitive or superior matching quality.

The data deduplication task has attracted a considerable amount of attention from the research community in order to provide effective and efficient solutions. The information provided by the user to tune the deduplication process is usually represented by a set of manually labeled pairs. In very large datasets, producing this kind of labeled set is a daunting task since it requires an expert to select and label a large number of informative pairs. In this article, we propose a two-stage sampling selection strategy (T3S) that selects a reduced set of pairs to tune the deduplication process in large datasets. T3S selects the most representative pairs by following two stages. In the first stage, we propose a strategy to produce balanced subsets of candidate pairs for labeling. In the second stage, an active selection is incrementally invoked to remove the redundant pairs in the subsets created in the first stage in order to produce an even smaller and more informative training set. This training set is effectively used both to identify where the most ambiguous pairs lie and to configure the classification approaches. Our evaluation shows that T3S is able to reduce the labeling effort substantially while achieving a competitive or superior matching quality when compared with state-of-the-art deduplication methods in large datasets.

Deduplication is the task of identifying which objects are potentially the same in a data repository. It usually demands user intervention in several steps of the process, mainly to identify some pairs representing matchings and non-matchings. This information is then used to help in identifying other potentially duplicated records. When deduplication is applied to very large datasets, the performance and matching quality depends on expert users to configure the most important steps of the process (e.g., blocking and classification). In this paper, we propose a new framework called FS-Dedup able to help tuning the deduplication process on large datasets with a reduced effort from the user, who is only required to label a small, automatically selected, subset of pairs. FS-Dedup exploits Signature-Based Deduplication (Sig-Dedup) algorithms in its deduplication core. Sig-Dedup is characterized by high efficiency and scalability in large datasets but requires an expert user to tune several parameters. FS-Dedup helps in solving this drawback by providing a framework that does not demand specialized user knowledge about the dataset or thresholds to produce high effectiveness. Our evaluation over large real and synthetic datasets (containing millions of records) shows that FS-Dedup is able to reach or even surpass the maximal matching quality obtained by Sig-Dedup techniques with a reduced manual effort from the user.

Service evolution is a critical issue because even small changes, if not compatible, can potentially affect a huge number of client applications. However, particularly in the context of large scale service usage, changes have different impact on clients according to its use. This paper proposes a change management framework that supports service providers to scope and quantify the impact of changes based on usage analysis. The framework adopts a finer-grained versioning model in order to easily locate and assess the compatibility of changes in service descriptions. The framework also clusters client applications based on similar patterns of usage, summarizing them in usage profiles. A usage profile quantifies the functionality of the service used by the corresponding applications, enabling to assess the impact of incompatible changes against the profile.

Services have been increasingly used as the building blocks for decoupled and flexible applications. Service evolution is a critical issue because even small changes, if not compatible, can potentially affect a huge number of client applications. However, particularly in the context of large scale usage of a service, changes cause different impact on client applications according to its use. This paper proposes to focus on compatibility from the point of view of usage patterns in order to deal with service evolution issues in more flexible and less costly way. The idea is to summarize the behavior of client applications into usage profiles, from which metrics that represent the impact of changes can be derived. This valuable information may support service providers on decisions about service lifecycle. The paper discusses the adoption of usage profiles and presents a framework for the automatic evaluation of service changes impact during its lifecycle.

(P2P) systems provide sharing of resources, which may be duplicated or versioned in several peers. However, traditional P2P systems are not aware of such replicas and versions, which arises an inefficiency and ineffectiveness problem. To solve this issue, our work proposes an automatic mechanism for replica and version detection. This functionality is presented as part of DetVX, an environment for managing replicas and versions of XML documents in a P2P context.

The problem of version detection is critical in many important application scenarios, including software clone identification, Web page ranking, plagiarism detection, and peer-to-peer searching. A natural and commonly used approach to version detection relies on analyzing the similarity between files. Most of the techniques proposed so far rely on the use of hard thresholds for similarity measures. However, defining a threshold value is problematic for several reasons: in particular (i) the threshold value is not the same when considering different similarity functions, and (ii) it is not semantically meaningful for the user. To overcome this problem, our work proposes a version detection mechanism for XML documents based on Naïve Bayesian classifiers. Thus, our approach turns the detection problem into a classification problem. In this paper, we present the results of various experiments on synthetic data that show that our approach produces very good results, both in terms of recall and precision measures.