The transactions of goods and services between enterprise service providers are often driven by contracts and purchase orders. Every month thousands of invoices are billed to customers who settle them based on the usage of services. Considering the vast number of purchase orders that are signed, it requires considerable manual effort by the service provider to process and manage them. Moreover, the invoice’s billed data may not be maintained in the same cloud system as the purchase orders. This leads to complexity with data mapping between the two data sets. Sometimes the invoices may get into a dispute due to over exhaustion of allocated funds or may be billed to an expired purchase order. Hence managing the billing service is a huge undertaking along with increased cost.To address these challenges, we developed an order manage- ment system that transforms the monitoring of purchase orders to increase renewals as well as decrease disputes. The system includes an automated purchase order-invoice data mapping model along with a risk analytics model that evaluates the orders against the invoices billed. The output is the set of actionable and non actionable insights based on customer portfolio, risk level as well as market trends in usage of services. We illustrate our method with some promising results on data of one of the world’s largest IT service providers.

When Information Technology (IT) service providers cater services to their customers, there’s a common practice to document the intent of buyer to purchase them through a Purchase Order (PO). The details of a PO document are complex in nature, as they include hierarchical structure of sub-services, price points over the duration of PO, base setup cost, billing frequency, renewal terms etc. In large enterprises with high volume PO’s, the traditional approach of managing PO’s with their invoices involved a great deal of inefficiency and labor-intensive manual work due to lack of automation and disconnected processing. As a result, they often suffered tedious manual monitoring and failed PO renewals, resulting in delays and added costs.In this paper, we propose a joint time-series learning framework that tackles the high dimensionality PO data. It identifies the metrics to monitor, while using time-series prediction coupled with tone analysis to reach out to customers, in order to maximise PO renewals. We demonstrate utility of our approach by implementing our method on a dataset from a global IT service provider that indicates encouraging results.

Sellers and executives who maintain a bidding pipeline of sales engagements with multiple clients for many opportunities significantly benefit from data-driven insight into the health of each of their bids. There are many predictive models that offer likelihood insights and win prediction modeling for these opportunities. Currently, these win prediction models are in the form of binary classification and only make a prediction for the likelihood of a win or loss. The binary formulation is unable to offer any insight as to why a particular deal might be predicted as a loss. This paper offers a multi-class classification model to predict win probability, with the three loss classes offering specific reasons as to why a loss is predicted, including no bid, customer did not pursue, and lost to competition. These classes offer an indicator of how that opportunity might be handled given the nature of the prediction. Besides offering baseline results on the multi-class classification, this paper also offers results on the model after class imbalance handling, with the results achieving a high accuracy of 85% and an average AUC score of 0.94.

In the face of emergent external factors (e.g., supply chain disruptions or public health crises like COVID-19), businesses must adapt their business model quickly in order to ensure service continuity. However, providing recommendations regarding changes should be made to the business model is a challenging problem. First, it requires details of interactions between different components of the business (e.g., service offerings, inventory, staffing, demand) to understand what possible courses of action will have the most business impact. Second, automated models may provide recommendations on changes required in the business operations. However, with lack of human insight, it will be hard to verify the feasibility of these recommendations. Third, a generic model may not be able to provide good recommendations for diverse set of business models. Fourth, the model may not have enough features or training data to provide good recommendations.In this paper, we propose a novel approach to provide actionable items that can be recommended to business users given their business features and recommendations given to businesses in similar domain. Here we first use clustering to find the business domain and similar feature set of the domain. Then, we build a machine-learning model with explainable insights to provide recommendations on different business actions that can be taken to ensure business operations in the face of emergent external factors. Next we augment our approach with human-in-the-loop to improve its performance. Finally, we federate the machine-learning model in a similar domain to add more explainable and trusted insights and recommendations by other businesses. We describe our method, illustrate its utility with results from our implementation, and discuss areas for future work.

Although widely used in various areas, the Deep Neural Network suffers from the lack of interpretability. Existing works usually focus on one data instance and the found explanations are thus limited. We argue that a good understanding of a model should contain both systematic explanations of model behavior and effective detection of its vulnerability. Particularly we propose to use backward deduction to achieve these two goals. Given a constraint on the model output, the deduction backtracks the architecture to find corresponding data ranges in the input. In each layer, depending on the type, specific rules and/or algorithms are developed. The resulted ranges in the end can be interpreted by sampling exemplary data from them. In experiment we show that with different strategies in selecting the data ranges, the sampled fake data can either explain the model or reflect its vulnerability.

Emotion analysis of online customer service conservation is important for good user experience and customer satisfaction. However, conventional metrics do not fit this application scenario. In this work, by collecting and labeling online conversations of customer service on Twitter, we identify 8 new metrics, named as tones, to describe emotional information. To better interpret each tone, we extend the Latent Dirichlet Allocation (LDA) model to Tone LDA (T-LDA). In T-LDA, each latent topic is explicitly associated with one of three semantic categories, i.e., tone-related, domain-specific and auxiliary. By integrating tone label into learning, T-LDA can interfere the original unsupervised training process and thus is able to identify representative tone-related words. In evaluation, T-LDA shows better performance than baselines in predicting tone intensity. Also, a case study is conducted to analyze each tone via T-LDA output.

Efficiency of large-scale learning is a hot topic in both academic and industry. The stochastic gradient descent (SGD) algorithm, and its extension mini-batch SGD, allow the model to be updated without scanning the whole data set. However, the use of approximate gradient leads to the uncertainty issue, slowing down the decreasing of objective function. Furthermore, such uncertainty may result in a high frequency of meaningless update on the model, causing a communication issue in parallel learning environment. In this work, we develop a batch-adaptive stochastic gradient descent (BA-SGD) algorithm, which can dynamically choose a proper batch size as learning proceeds. Particularly on the basis of Taylor extension and central limit theorem, it models the decrease of objective value as a Gaussian random walk game with rebound. In this game, a heuristic strategy of determining batch size is adopted to maximize the utility of each incremental sampling. By evaluation on multiple real data sets, we demonstrate that by smartly choosing the batch size, the BA-SGD not only conserves the fast convergence of SGD algorithm but also avoids too frequent model updates.

In IT strategic outsourcing businesses, it is critical to have competent deal teams design competitive service solutions and swiftly respond to clients' requests for proposals. In this paper we present a general team recommendation framework for finding the best deal teams to pursue such engagement opportunities. Little previous work on team recommendations considers both individual and team-level features at the same time. Our proposed framework can take into account diverse individual and team features, and accommodate various cost or feature functions. We introduce a team quality metric based on a weighted linear combination of these features, the weights of which are learned using a machine learning approach by leveraging historical project outcomes. A combinatorial optimization algorithm is finally applied to search the possible solution space for the approximate best team. We report a preliminary evaluation of our framework by applying it to real-world data from strategic outsourcing businesses at a large IT service company. We also compare our approach with other existing work by using the public DBLP dataset for recommending teams in academic paper authoring.

This paper describes a novel framework for creating a parser to process and analyze texts written in a “partially structured” natural language. In many projects, the contents of document artifacts tend to be described as a mixture of formal parts (i.e. the text constructs follow specific conventions) and parts written in arbitrary free text. Formal parsers, typically defined and used to process a description with rigidly defined syntax such as program source code are very precise and efficient in processing the formal part, while parsers developed for natural language processing (NLP) are good at robustly interpreting the free-text part. Therefore, combining these parsers with different characteristics can allow for more flexible and practical processing of various project documents. Unfortunately, conventional approaches to constructing a parser from multiple small parsers were studied extensively only for formal language parsers and are not directly applicable to NLP parsers due to the differences in the way the input text is extracted and evaluated. We propose a method to configure and generate a combined parser by extending an approach based on parser combinator, the operators for composing multiple formal parsers, to support both NLP and formal parsers. The resulting text parser is based on Parsing Expression Grammars, and it benefits from the strength of both parser types. We demonstrate an application of such combined parser in practical situations and show that the proposed approach can efficiently construct a parser for analyzing project-specific industrial specification documents.

We describe a new approach to modeling and analyzing software development documents that are typically written using conventional office applications. Our approach brings automation to content extraction, quality checking and measurement of massive document artifacts that tend to be handled by labor-intensive manual work in industry today. Rather than seeking an approach based on creation or rewriting of contents using more rigid, machine-friendly representations such as standardized formal models and restricted languages, we provide a method to deal with the diversity of document artifacts by making use of project-specific formalism that exists in target documents. We demonstrate that such project-specific formalism often tends to "naturally" exist at syntactic levels, and it is possible to define a "document model", a logical data representation gained by transformation rule from the physical, syntactic structure to the logical, semantic structure. With this transformation, various quality checking rules for completeness, consistency, traceability, etc., are realized by evaluating constraints for data items in the logical structure, and measurement of these quality aspects is automated. We developed a tool to allow a user to easily define document models and checking rules, and provide the insights on transformations when defining document models for various industry specification documents written in word processor files, spreadsheets and presentations. We also demonstrate the use of natural language processing can improve document modeling and quality checking by compensating for a weakness of formalism and applying analysis to specific parts of the target documents.

Natural language is the preferred form for writing use cases. While a few linguistic techniques exist that extract or validate structured information from unstructured natural language use case, they often cannot be extended beyond their primary language. Extending linguistic analysis and automated validation capabilities across multiple languages is necessary not only for widespread industrial adoption but it helps in analyzing a collection of multilingual use cases (quite frequent in multi-national projects) that need to be aggregated. We have published a UIMA (Unstructured Information Management Architecture) based linguistic engine for analyzing English use cases. In this paper, we report on extension of our linguistic technique to Japanese and effect of such an extension on the automated requirement validation suite.

We present the design and implementation of HPCBugBase, an experience base for high performance computing (HPC) software defects. Our goal is to accumulate empirical knowledge about commonly occurring defects in HPC codes using an incremental approach. This knowledge is structured so that HPC practitioners such as programmers and tool builders can use it to reduce debugging costs, as well as provide feedback which becomes incorporated into the system. By building the experience base, we expect to help the process of making explicit the knowledge about recurring defects that otherwise cannot be shared. The current system is built on a Wiki system, which allows incremental accumulation of data at various levels of abstraction. We implement additional analysis functions that do not exist in a generic Wiki system as custom plug-ins. We have populated the system with data collected from software engineering studies from the DARPA High Productivity Computer Systems Project.

We present the design and implementation of HPCBugBase, an experience base for high performance computing (HPC) software defects. Our goal is to accumulate empirical knowledge about commonly occurring defects in HPC codes using an incremental approach. This knowledge is structured so that HPC practitioners such as programmers and tool builders can use it to reduce debugging costs, as well as provide feedback which becomes incorporated into the system. By building the experience base, we expect to help the process of making explicit the knowledge about recurring defects that otherwise cannot be shared. The current system is built on a Wiki system, which allows incremental accumulation of data at various levels of abstraction. We implement additional analysis functions that do not exist in a generic Wiki system as custom plug-ins. We have populated the system with data collected from software engineering studies from the DARPA High Productivity Computer Systems Project.