Chatbots are widely employed in various scenarios. However, given the high costs of chatbot development and chatbots’ tremendous social influence, chatbot failures may inevitably lead to a huge economic loss. Previous chatbot evaluation frameworks rely heavily on human evaluation, lending little support for automatic early-stage chatbot examination prior to deployment. To reduce the risk of potential loss, we propose a computational approach to extracting features and training models that make a priori prediction about chatbots’ popularity, which indicates chatbot general performance. The features we extract cover chatbot Intent, Conversation Flow, and Response Design. We studied 1050 customer service chatbots on one of the most popular chatbot service platforms. Our model achieves 77.36% prediction accuracy among very popular and very unpopular chatbots, making the first step towards computational feedback before chatbot deployment. Our evaluation results also reveal the key design features associated with chatbot popularity and offer guidance on chatbot design.

Social media users are overloaded with diverse opinions by people with opposing stances. Previous research shows that people often look for opinions that reinforce their pre-existing beliefs and stances, which may lead to social polarization. Traditional social media present opinions in a linear list format, which not only lacks structures for people to explore diverse viewpoints but also aggravates their selective exposure to agreeable opinions. To address this problem, we designed an intelligent system that improves awareness of diverse social opinions by providing visual hints and recommendations of opinions (e.g. news articles and comments) on different sides with different indicators. We evaluated our system with news articles about Obamacare repeal issue and their corresponding user comments from Facebook. Results demonstrate that our system could increase people»s awareness of their stances and opinion selection preferences, which mitigates selective exposure and thereby leads to a more balanced perception of social opinions.

Social media sites use different labels to help users find and select news feeds. For example, Blue Feed, Red Feed, a news feed created by the Wall Street Journal, use stance labels to separate news articles with opposing political ideologies to help people explore diverse opinions. To combat the spread of fake news, Facebook has experimented with putting credibility labels on news articles to help readers decide whether the content is trustworthy. To systematically understand the effects of stance and credibility labels on online news selection and consumption, we conducted a controlled experiment to study how these labels influence the selection, perceived extremeness, and level of agreement of news articles. Results show that stance labels may intensify selective exposure - a tendency for people to look for agreeable opinions -- and make people more vulnerable to polarized opinions and fake news. We found, however, that the effect of credibility labels on reducing selective exposure and recognizing fake news is limited. Although originally designed to encourage exposure to opposite viewpoints, stance labels can make fake news articles look more trustworthy, and they may lower people's perception of the extremeness of fake news articles. Our results have important implications on the subtle effects of stance and credibility labels on online news consumption.

An enormous amount of posts and comments are shared in online social forums, which often organize these online social opinions based on semantic contents. However, for controversial topics, people with different attitudes and stances often have very distinct perspectives, reactions, and emotions to the same post. Organization by semantic contents often encourages selective exposure to information, which may exacerbate opinion polarization. To address this problem, we design a novel interface that allows people to better understand and appreciate people with different stances in social forums. Our interface was developed to allow interactive visualization and categorization of original posts about a controversial topic with crowd workers' reactions and emotions from different stances. We evaluated the interface using Reddit posts about US presidential candidates. Results demonstrate that the interface can mitigate selective exposure and help users to adopt a broader spectrum of opinions than the traditional Reddit interface.

Crowdsourcing is a viable mechanism for creating training data for machine translation.It provides a low cost, fast turnaround way of processing large volumes of data.However, when compared to professional translation, naive collection of translations from non-professionals yields low-quality results.Careful quality control is necessary for crowdsourcing to work well.In this paper, we examine the challenges of a two-step collaboration process with translation and post-editing by non-professionals.We develop graphbased ranking models that automatically select the best output from multiple redundant versions of translations and edits, and improves translation quality closer to professionals.