In this work, we propose a Weakly Semantic-based approach for Sentiment Analysis (WSSA), a novel approach that analyzes sentiment by considering weak labels from different sources (sentiment analysis tools) and aggregates them based on features such as the consistency between sources, the semantic equivalence between documents, and experts’ domain knowledge in order to improve the sentiments analysis tools results. The aggregation is achieved using a Probabilistic Soft Logic reasoner to infer the documents’ polarity.

Sentiment analysis has received constant research attention due to its usefulness and importance in different applications. However, despite the research advances in this field, most current tools suffer in prediction quality due to the inconsistencies in their results, i.e., intra- and inter-tool inconsistencies. This demonstration proposes a system for the evaluation of sentiment analysis quality namely SA-Q. The system allows the evaluation of inconsistency in sentiment analysis tools, the resolution of the inconsistency using state-of-the-art methods and the recommendation of relevant sentiment analysis tool for any type of data set provided by the attendees. It allows the attendees to compare the tools. Moreover, we demonstrate that SA-Q evaluates the consistency of tools on two levels (intra-tool and inter-tool). Through various scenarios, we showcase the challenges of inconsistency resolution, demonstrate the usefulness of the proposed system and the recommendations that can be given to the attendees for their datasets. We demonstrate that SA-Q system has practical utility in many areas of industrial applications for better decision making. This demonstration shows promising research areas for data management, NLP, and machine learning communities by adopting and drawing inspiration from truth inference methods to create more robust tools and improve the tool's scalability.

In this paper, we present a comprehensive study that evaluates six state-of-the-art sentiment analysis tools on five public datasets, based on the quality of predictive results in the presence of semantically equivalent documents, i.e., how consistent existing tools are in predicting the polarity of documents based on paraphrased text. We observe that sentiment analysis tools exhibit intra-tool inconsistency , which is the prediction of different polarity for semantically equivalent documents by the same tool, and inter-tool inconsistency , which is the prediction of different polarity for semantically equivalent documents across different tools. We introduce a heuristic to assess the data quality of an augmented dataset and a new set of metrics to evaluate tool inconsistencies. Our results indicate that tool inconsistencies is still an open problem, and they point towards promising research directions and accuracy improvements that can be obtained if such inconsistencies are resolved.

Crowdsourcing is emerging as a powerful paradigm to help perform a wide range of tedious tasks in various enterprise applications. As such applications become more complex, crowdsourcing systems often require the collaboration of several experts connected through professional/social networks and organized in various teams. For instance, a well-known car manufacturer asked fans to contribute ideas for the kinds of technologies that should be incorporated into one of its cars. For that purpose, fans needed to collaborate and form teams competing with each others to come up with the best ideas. However, once teams are formed, each one would like to provide the best solution and treat that solution as a “trade secret,” hence preventing any data leak to its competitors (i.e., the other teams). In this article, we propose a data leak--aware crowdsourcing system called SocialCrowd . We introduce a clustering algorithm that uses social relationships between crowd workers to discover all possible teams while avoiding interteam data leakage. We also define a ranking mechanism to select the “best” team configurations. Our mechanism is based on the semiring approach defined in the area of soft constraints programming. Finally, we present experiments to assess the efficiency of the proposed approach.

Crowd sourcing is emerging as a powerful paradigm to solve a wide range of tedious and complex problems in various enterprise applications. It spawns the issue of finding the unknown collaborative and competitive group of solvers. The formation of collaborative team should provide the best solution and treat that solution as a trade secret avoiding data leak between competitive teams due to reward behind the outsourcing of the issue. The formation of effective competitive teams not only requires adequate skills between members of each team, but also good team connectivity through social network and to provide the best solution and treat that solution as a trade secret avoiding data leak between teams due to reward behind the outsourcing of the issue. In this paper, we propose a data leak aware crowd sourcing system called Social Crowd. We introduce a clustering algorithm that uses social relationships between crowd workers to discover all possible teams while avoiding inter-team data leakage.