Chatbots are widely employed in various scenarios. However, given the high costs of chatbot development and chatbots’ tremendous social influence, chatbot failures may inevitably lead to a huge economic loss. Previous chatbot evaluation frameworks rely heavily on human evaluation, lending little support for automatic early-stage chatbot examination prior to deployment. To reduce the risk of potential loss, we propose a computational approach to extracting features and training models that make a priori prediction about chatbots’ popularity, which indicates chatbot general performance. The features we extract cover chatbot Intent, Conversation Flow, and Response Design. We studied 1050 customer service chatbots on one of the most popular chatbot service platforms. Our model achieves 77.36% prediction accuracy among very popular and very unpopular chatbots, making the first step towards computational feedback before chatbot deployment. Our evaluation results also reveal the key design features associated with chatbot popularity and offer guidance on chatbot design.

Given the increasing popularity of customer service dialogue on Twitter, analysis of conversation data is essential to understanding trends in customer and agent behavior for the purpose of automating customer service interactions. In this work, we develop a novel taxonomy of fine-grained “dialogue acts” frequently observed in customer service, showcasing acts that are more suited to the domain than the more generic existing taxonomies. Using a sequential SVM-HMM model, we model conversation flow, predicting the dialogue act of a given turn in real time, and showcase this using our “PredDial” portal. We characterize differences between customer and agent behavior in Twitter customer service conversations and investigate the effect of testing our system on different customer service industries. Finally, we use a data-driven approach to predict important conversation outcomes: customer satisfaction, customer frustration, and overall problem resolution. We show that the type and location of certain dialogue acts in a conversation have a significant effect on the probability of desirable and undesirable outcomes and present actionable rules based on our findings. We explore the correlations between different dialogue acts and the outcome of the conversations in detail using an actionable-rule discovery task by leveraging a state-of-the-art sequential rule mining algorithm while modeling a set of conversations as a set of sequences. The patterns and rules we derive can be used as guidelines for outcome-driven automated customer service platforms.

Chatbot has become an important solution to rapidly increasing customer care demands on social media in recent years. However, current work on chatbot for customer care ignores a key to impact user experience - tones. In this work, we create a novel tone-aware chatbot that generates toned responses to user requests on social media. We first conduct a formative research, in which the effects of tones are studied. Significant and various influences of different tones on user experience are uncovered in the study. With the knowledge of effects of tones, we design a deep learning based chatbot that takes tone information into account. We train our system on over 1.5 million real customer care conversations collected from Twitter. The evaluation reveals that our tone-aware chatbot generates as appropriate responses to user requests as human agents. More importantly, our chatbot is perceived to be even more empathetic than human agents.

Successful human interactions are based on becoming aware of other's emotion and making adaptations accordingly. However, understanding emotion is a complex task that has generated countless debates among researchers over the past decades. The abstract nature of human emotion highlights the need for a new data-driven approach that can better describe and compare across fine-grained emotional states. In this study, we propose Seemo, a novel neural embedding framework, which allows us to map human emotions into vector space representations. Seemo is trained using Twitter data and is evaluated on two fundamental use cases in traditional emotion research: determining the underlying dimensions of emotions and identifying the set of basic emotions. The evaluation reveals that on both tasks Seemo can generate results consistent with the mainstream theories. Results also show that the vector space representation of Seemo can effectively decode the important relationships between emotions that were usually not explicitly presented.

Users are rapidly turning to social media to request and receive customer service; however, a majority of these requests were not addressed timely or even not addressed at all. To overcome the problem, we create a new conversational system to automatically generate responses for users requests on social media. Our system is integrated with state-of-the-art deep learning techniques and is trained by nearly 1M Twitter conversations between users and agents from over 60 brands. The evaluation reveals that over 40% of the requests are emotional, and the system is about as good as human agents in showing empathy to help users cope with emotional situations. Results also show our system outperforms information retrieval system based on both human judgments and an automatic evaluation metric.

Given the increasing popularity of customer service dialogue on Twitter, analysis of conversation data is essential to understand trends in customer and agent behavior for the purpose of automating customer service interactions. In this work, we develop a novel taxonomy of fine-grained "dialogue acts" frequently observed in customer service, showcasing acts that are more suited to the domain than the more generic existing taxonomies. Using a sequential SVM-HMM model, we model conversation flow, predicting the dialogue act of a given turn in real-time. We characterize differences between customer and agent behavior in Twitter customer service conversations, and investigate the effect of testing our system on different customer service industries. Finally, we use a data-driven approach to predict important conversation outcomes: customer satisfaction, customer frustration, and overall problem resolution. We show that the type and location of certain dialogue acts in a conversation have a significant effect on the probability of desirable and undesirable outcomes, and present actionable rules based on our findings. The patterns and rules we derive can be used as guidelines for outcome-driven automated customer service platforms.

In IT strategic outsourcing businesses, it is critical to have competent deal teams design competitive service solutions and swiftly respond to clients' requests for proposals. In this paper we present a general team recommendation framework for finding the best deal teams to pursue such engagement opportunities. Little previous work on team recommendations considers both individual and team-level features at the same time. Our proposed framework can take into account diverse individual and team features, and accommodate various cost or feature functions. We introduce a team quality metric based on a weighted linear combination of these features, the weights of which are learned using a machine learning approach by leveraging historical project outcomes. A combinatorial optimization algorithm is finally applied to search the possible solution space for the approximate best team. We report a preliminary evaluation of our framework by applying it to real-world data from strategic outsourcing businesses at a large IT service company. We also compare our approach with other existing work by using the public DBLP dataset for recommending teams in academic paper authoring.

Service-oriented way of building customizations to packaged applications is an emerging alternative to the traditional way of modifying the package application directly to implement customizations. Estimating the effort at an early stage (before detailed design) in the services engagement is important to reduce the perceived risk of the new approach and to demonstrate the potential cost efficiencies. In this paper we present a method to estimate the effort and cost involved in developing customizations to packaged application software (such as SAP and Oracle ERP software) using service-oriented architecture/design (SOA) style. Taking only a description of business processes to be customized, we estimate the effort and costs of implementing those customizations. We use novel artifact-centric and linguistic analysis approaches to estimate the business object count which lies at the center of our cost estimation model. The model is currently being piloted at a large IT services organization involved in implementing packaged application software for clients. Initial experiments reveal that the cost and effort estimates are within 10% range of the benchmark estimates developed after requirements gathering and detailed design.

While the traditional model driven development techniques are useful for building solutions in a reusable manner, they do not say much about how the existing assets in a client environment can be leveraged effectively and efficiently. In this work, we enhance model driven transformation techniques to generate implementation artifacts on a given platform from platform independent models while leveraging the existing assets in a client environment. We apply semantic Web service matching technology to achieve automatic binding of generated artifacts with available client assets. By generating implementation artifacts that are bound where appropriate with clientspsila existing functionality, our approach helps cut down the development time during project implementations and thereby resulting in reduced project durations and costs. We demonstrate the feasibility of the two platforms: IBM WebSphere and SAP NetWeaver. Lessons learned are presented.

Mashup editors, like Yahoo Pipes and IBM Lotus Mashup Maker, allow non-programmer end-users to ldquomash-uprdquo information sources and services to meet their information needs. However, with the increasing number of services, information sources and complex operations like filtering and joining, even an easy to use editor is not sufficient. MashupAdvisor aims to assist mashup creators to build higher quality mashups in less time. Based on the current state of a mashup, the MashupAdvisor quietly suggests outputs (goals) that the user might want to include in the final mashup. MashupAdvisor exploits a repository of mashups to estimate the popularity of specific outputs, and makes suggestions using the conditional probability that an output will be included, given the current state of the mashup. When a suggestion is accepted, MashupAdvisor uses a semantic matching algorithm and a metric planner to modify the mashup to produce the suggested output. Our prototype was implemented on top of IBM Lotus MashupMaker and our initial results show that it is effective.

Automatically finding suitable Web services given a request is a difficult problem because the interface descriptions of Web services are often terse and cryptic. Dictionary and information retrieval based techniques have proven useful in disambiguating the semantics of service descriptions, but they are limited in their capability to consider the relationships between the words describing the Web services. Current ontology-based approaches typically require a user to explicitly create domain ontologies. This paper presents a novel technique that significantly improves the quality of semantic Web service matching by (1) automatically generating ontologies based on Web service descriptions and (2) using these ontologies to guide the mapping between Web services. Our approach differs from earlier work on service matching by considering the relationship between words rather than treating them as a bag of unrelated words. The experimental results indicate that with our unsupervised approach we can eliminate up to 70% of incorrect matches that are made by dictionary-based approaches.

This paper presents a novel technique that significantly improves the quality of semantic Web service matching by (1) automatically generating ontologies based on Web service descriptions and (2) using these ontologies to guide the mapping between Web services.

In this paper, we present a novel algorithm to compose Web services in the presence of semantic ambiguity by combining semantic matching and AI planning algorithms. Specifically, we use cues from domain-independent and domain-specific ontologies to compute an overall semantic similarity score between ambiguous terms. This semantic similarity score is used by AI planning algorithms to guide the searching process when composing services. Experimental results indicate that planning with semantic matching produces better results than planning or semantic matching alone. The solution is suitable for semi-automated composition tools or directory browsers

In this paper, we explore the use of domain-independent and domain-specific ontologies to find matching service descriptions. The domain-independent relationships are derived using an English thesaurus after tokenization and part-of-speech tagging. The domain-specific ontological similarity is derived by an inference on the semantic annotations associated with Web service descriptions. Matches due to the two cues are combined to determine an overall semantic similarity score. By combining multiple cues, we show that better relevancy results can be obtained for service matches from a large repository, than could be obtained using any one cue alone.

As an industry-backed registry for Web Services, UDDI plays an important role in helping requesters find suitable services. Unfortunately, the current search functions in UDDI are limited in their support for making automatic service selection decisions. While some approaches have been suggested to enhance the search capabilities in UDDI using service semantics, they suffer from limitations. These approaches either prescribe performing semantic matching outside of UDDI leaving its search function unchanged or propose embedding a specific matching engine within UDDI thereby making the search function inflexible. In this work, we present a flexible mechanism to enhance UDDI search function. Using our approach, users can integrate multiple external matching services with a UDDI registry to support multiple external service description languages. The result is a UDDI registry with flexible and intelligent service search function that can be used for dynamic service selection.

The advent of Web services has made automated workflow composition relevant to Web based applications. One technique, that has received some attention, for automatically composing workflows is AI-based classical planning. However, classical planning suffers from the paradox of first assuming deterministic behavior of Web services, then requiring the additional overhead of execution monitoring to recover from unexpected behavior of services. To address these concerns, we propose using Markov decision processes (MDPs), to model workflow composition. Our method models both, the inherent stochastic nature of Web services, and the dynamic nature of the environment. The resulting workflows are robust to nondeterministic behaviors of Web services and adaptive to a changing environment. Using an example scenario, we demonstrate our method and provide empirical results in its support.