Current datasets for unwanted social bias auditing are limited to studying protected demographic features such as race and gender. In this work, we introduce a comprehensive benchmark that is meant to capture the amplification of social bias, via stigmas, in generative language models. Taking inspiration from social science research, we start with a documented list of 93 US-centric stigmas and curate a question-answering (QA) dataset which involves simple social situations. Our benchmark, SocialStigmaQA, contains roughly 10K prompts, with a variety of prompt styles, carefully constructed to systematically test for both social bias and model robustness. We present results for SocialStigmaQA with two open source generative language models and we find that the proportion of socially biased output ranges from 45% to 59% across a variety of decoding strategies and prompting styles. We demonstrate that the deliberate design of the templates in our benchmark (e.g., adding biasing text to the prompt or using different verbs that change the answer that indicates bias) impacts the model tendencies to generate socially biased output. Additionally, through manual evaluation, we discover problematic patterns in the generated chain-of-thought output that range from subtle bias to lack of reasoning. Warning: This paper contains examples of text which are toxic, biased, and potentially harmful.

As artificial intelligence and machine learning algorithms become increasingly prevalent in society, multiple stakeholders are calling for these algorithms to provide explanations. At the same time, these stakeholders, whether they be affected citizens, government regulators, domain experts, or system developers, have different explanation needs. To address these needs, in 2019, we created AI Explainability 360, an open source software toolkit featuring ten diverse and state-of-the-art explainability methods and two evaluation metrics. This paper examines the impact of the toolkit with several case studies, statistics, and community feedback. The different ways in which users have experienced AI Explainability 360 have resulted in multiple types of impact and improvements in multiple metrics, highlighted by the adoption of the toolkit by the independent LF AI & Data Foundation. The paper also describes the flexible design of the toolkit, examples of its use, and the significant educational material and documentation available to its users.

Clinical notes are becoming an increasingly important data source for machine learning (ML) applications in healthcare. Prior research has shown that deploying ML models can perpetuate existing biases against racial minorities, as bias can be implicitly embedded in data. In this study, we investigate the level of implicit race information available to ML models and human experts and the implications of model-detectable differences in clinical notes. Our work makes three key contributions. First, we find that models can identify patient self-reported race from clinical notes even when the notes are stripped of explicit indicators of race. Second, we determine that human experts are not able to accurately predict patient race from the same redacted clinical notes. Finally, we demonstrate the potential harm of this implicit information in a simulation study, and show that models trained on these race-redacted clinical notes can still perpetuate existing biases in clinical treatment decisions.

Interpretable and explainable machine learning has seen a recent surge of interest. We focus on safety as a key motivation behind the surge and make the relationship between interpretability and safety more quantitative. Toward assessing safety, we introduce the concept of maximum deviation via an optimization problem to find the largest deviation of a supervised learning model from a reference model regarded as safe. We then show how interpretability facilitates this safety assessment. For models including decision trees, generalized linear and additive models, the maximum deviation can be computed exactly and efficiently. For tree ensembles, which are not regarded as interpretable, discrete optimization techniques can still provide informative bounds. For a broader class of piecewise Lipschitz functions, we leverage the multi-armed bandit literature to show that interpretability produces tighter (regret) bounds on the maximum deviation. We present case studies, including one on mortgage approval, to illustrate our methods and the insights about models that may be obtained from deviation maximization.

This paper addresses the task of explaining anomalous predictions of a black-box regression model. When using a black-box model, such as one to predict building energy consumption from many sensor measurements, we often have a situation where some observed samples may significantly deviate from their prediction. It may be due to a sub-optimal black-box model, or simply because those samples are outliers. In either case, one would ideally want to compute a responsibility score indicative of the extent to which an input variable is responsible for the anomalous output. In this work, we formalize this task as a statistical inverse problem: Given model deviation from the expected value, infer the responsibility score of each of the input variables. We propose a new method called likelihood compensation (LC), which is founded on the likelihood principle and computes a correction to each input variable. To the best of our knowledge, this is the first principled framework that computes a responsibility score for real valued anomalous model deviations. We apply our approach to a real-world building energy prediction task and confirm its utility based on expert feedback.

As artificial intelligence algorithms make further inroads in high-stakes societal applications, there are increasing calls from multiple stakeholders for these algorithms to explain their outputs. To make matters more challenging, different personas of consumers of explanations have different requirements for explanations. Toward addressing these needs, we introduce AI Explainability 360, an open-source Python toolkit featuring ten diverse and state-of-the-art explainability methods and two evaluation metrics. Equally important, we provide a taxonomy to help entities requiring explanations to navigate the space of interpretation and explanation methods, not only those in the toolkit but also in the broader literature on explainability. For data scientists and other users of the toolkit, we have implemented an extensible software architecture that organizes methods according to their place in the AI modeling pipeline. The toolkit is not only the software, but also guidance material, tutorials, and an interactive web demo to introduce AI explainability to different audiences. Together, our toolkit and taxonomy can help identify gaps where more explainability methods are needed and provide a platform to incorporate them as they are developed.

Autonomous cyber-physical agents play an increasingly large role in our lives. To ensure that they behave in ways aligned with the values of society, we must develop techniques that allow these agents to not only maximize their reward in an environment, but also to learn and follow the implicit constraints of society. We detail a novel approach that uses inverse reinforcement learning to learn a set of unspecified constraints from demonstrations and reinforcement learning to learn to maximize environmental rewards. A contextual bandit-based orchestrator then picks between the two policies: constraint-based and environment reward-based. The contextual bandit orchestrator allows the agent to mix policies in novel ways, taking the best actions from either a reward-maximizing or constrained policy. In addition, the orchestrator is transparent on which policy is being employed at each time step. We test our algorithms using Pac-Man and show that the agent is able to learn to act optimally, act within the demonstrated constraints, and mix these two functions in complex ways.

The United Nations Development Programme (UNDP) helps countries implement the United Nations (UN) Sustainable Development Goals (SDGs), an agenda for tackling major societal issues such as poverty, hunger, and environmental degradation by the year 2030. A key service provided by UNDP to countries that seek it is a review of national development plans and sector strategies by policy experts to assess alignment of national targets with one or more of the 169 targets of the 17 SDGs. Known as the Rapid Integrated Assessment (RIA), this process involves manual review of hundreds, if not thousands, of pages of documents and takes weeks to complete. In this work, we develop a natural language processing-based methodology to accelerate the workflow of policy experts. Specifically we use paragraph embedding techniques to find paragraphs in the documents that match the semantic concepts of each of the SDG targets. One novel technical contribution of our work is in our use of historical RIAs from other countries as a form of neighborhood-based supervision for matches in the country under study. We have successfully piloted the algorithm to perform the RIA for Papua New Guinea’s national plan, with the UNDP estimating it will help reduce their completion time from an estimated 3-4 weeks to 3 days.

This paper proposes an approach to estimating fungibility between skills given multiple information sources of those skills. An estimate of skill adjacency or fungibility or substitutability is critical for effective capacity planning, analytics and optimization in the face of changing skill requirements of an organization. The proposed approach is based on computing a similarity measure between skills, using each available data source, and combining these similarities into a measure of fungibility. We present both supervised and unsupervised integration methods and demonstrate that these produce improved outcomes, compared to using any single skill similarity source alone, using data from a large IT organization. The skills’ fungibility matrix created using this approach has been deployed by the organization for demand forecasting across groups of skills. We discuss how the fungibility matrix is deployed to generate skill clusters and present a forecasting algorithm that additionally incorporates past/future engagements and a mechanism to quantify uncertainty in the forecast. A possible extension of this work is to use the fungibility measure to cluster skills and develop a skill-centric representation of an organization to enable strategic assessments and planning.

Predictive modeling is becoming increasingly common in healthcare. Existing healthcare cost prediction solutions are tailor-made to accomplish specific tasks for certain populations, hence requiring expensive modifications to adapt to a different task or population. In this paper, we present a modular and extensible solution for healthcare cost prediction, which can be easily configured for various prediction tasks and populations. Our solution incorporates efficient high-dimensional data handling, smart feature engineering, flexible predictive learning, individualized assessment of cost impacts of predictors, and a management system that allows for reuse of partial results. We configure two distinct applications using the proposed system and present results on prediction accuracy and cost impact assessment. The first application predicts healthcare costs for a commercial population, and the second predicts the cost of care for a Medicaid population using an entirely different set of data, predictors, and assumptions.

Modern organizations invest a lot of resources in recruiting, managing, and retaining people with high value and talent. In spite of several studies over the past fifty years, there is no silver bullet for talent management, since the area itself is constantly evolving due to the ever-changing nature of the enterprise in the knowledge economy. In this paper, we adopt an analytics-based approach to advancing talent management, particularly from the point of view of employee commitment. An individual employee's commitment is quantified using her propensity to leave the company, which is modeled using historical employee records and other organization-specific data. Furthermore, factors behind this predicted level of commitment are also identified using data mining approaches. The predictive modeling is made robust and actionable by paying special attention to the accuracy of the propensity scores, their stability over time, and the inter-pretability of the factors. The propensity scores and identified factors are used to infer meaningful recommendations that are helpful to an employee's career apart from being consistent with the business objectives of the organization. We have incorporated all of this in a talent management tool which is an integrated platform for all stakeholders — employees, managers, top-line management and human resource professionals. This tool has been deployed in a large, global, Fortune 500 organization for about 100,000 employees. The results of the deployment are very promising with significant tangible monetary benefits, as well as possible intangible benefits such as improved awareness of the management on factors behind employee commitment, increased communication of employees with the management, and improved employee engagement.

Signals in response to time-localized events of a common phenomenon tend to exhibit a common shape, but with variable time scale, amplitude, and delay across trials in many domains. We develop a new formulation to learn the common shape and variables from noisy signal samples with a Bayesian signal model and a Markov chain Monte Carlo inference scheme involving Gibbs sampling and independent Metropolis-Hastings. Our experiments with generated and real-world data show that the algorithm is robust to missing data, outperforms the existing approaches and produces easily interpretable outputs.

Many applications require matching objects to a predefined, yet highly dynamic set of categories accompanied by category descriptions. We present a novel approach to solving this class of categorization problems by formulating it in a semi-supervised clustering framework. Text-based matching is performed to generate ldquosoftrdquo seeds, which are then used to guide clustering in the basic feature space. We introduce a new variation of the k-means algorithm, called Soft Seeded k-means, which can effectively incorporate seeds that are of varying degrees of confidence, while allowing for incomplete coverage of the pre-defined categories. The algorithm is applied to real-world data from a business analytics application, and we demonstrate that it leads to superior performance compared to previous approaches.

We present a new L1-distance-based k-means clustering algorithm to address the challenge of clustering high-dimensional proportional vectors. The new algorithm explicitly incorporates proportionality constraints in the computation of the cluster centroids, resulting in reduced L1 error rates. We compare the new method to two competing methods, an approximate L1-distance k-means algorithm, where the centroid is estimated using cluster means, and a median L1 k-means algorithm, where the centroid is estimated using cluster medians, with proportionality constraints imposed by normalization in a second step. Application to clustering of projects based on distribution of labor hours by skill illustrates the advantages of the new algorithm.

The development of an aggregate view of the procurement spend across an enterprise using transactional data is increasingly becoming a very important and strategic activity. Not only does it provide a complete and accurate picture of what the enterprise is buying and from whom, it also allows it to consolidate suppliers, as well as negotiate better prices. The importance, as well as the complexity, of this cleansing exercise is further magnified by the increasing popularity of Business Transformation Outsourcing (BTO) wherein enterprises are turning over non-core activities, such as indirect procurement, to third parties, who now need to develop an integrated view of spend across multiple enterprises in order to optimize procurement and generate maximum savings. However, the creation of such an integrated view of procurement spend requires the creation of a homogeneous data repository from disparate (heterogeneous) data sources across various geographic and functional organizations throughout the enterprise(s). Such repositories get transactional data from various sources such as invoices, purchase orders, account ledgers. As such, the transactions are not cross-indexed, refer to the same suppliers by different names, and use different ways of representing information about the same commodities. Before an aggregated spend view can be developed, this data needs to be cleansed, primarily to normalize the supplier names and correctly map each transaction to the appropriate commodity code. Commodity mapping, in particular, is made more difficult by the fact that it has to be done on the basis of unstructured text descriptions found in the various data sources. We describe an on-demand system to automatically perform this cleansing activity using techniques from information retrieval and machine learning. Built on standard integration and application infrastructure software, this system provides enterprises with a fast, reliable, accurate and on-demand way of cleansing transactional data and generating an integrated view of spend. This system is currently in the process of being deployed by IBM for use in its BTO practice.

Part numbers are widely used within an enterprise throughout the manufacturing process. The point of entry of such part numbers into this process is normally via a Bill of Materials, or BOM, sent by a contact manufacturer or supplier. Each line of the BOM provides information about one part such as the supplier part number, the BOM receiver's corresponding internal part number, an unstructured textual part description, the supplier name, etc. However, in a substantial number of cases, the BOM receiver's internal part number is absent. Hence, before this part can be incorporated into the receiver's manufacturing process, it has to be mapped to an internal part (of the BOM receiver) based on the information of the part in the BOM. Historically, this mapping process has been done manually which is a highly time-consuming, labor intensive and error-prone process. This paper describes a system for automating the mapping of BOM part numbers. The system uses a two step modeling and mapping approach. First, the system uses historical BOM data, receiver's part specifications data and receiver's part taxonomic data along with domain knowledge to automatically learn classification models for mapping a given BOM part description to successively lower levels of the receiver's part taxonomy to reduce the set of potential internal parts to which the BOM part could map to. Then, information about various part parameters is extracted from the BOM part description and compared to the specifications data of the potential internal parts to choose the final mapped internal part. Mappings done by the system are very accurate, and the system is currently being deployed within IBM for mapping BOMs received by the corporate procurement/manufacturing divisions.

We are building a collaborative learning environement that supports teams of students as they collaborate synchronously and remotely to solve situated, multi-step problems involving algebraic modeling. Our system, named Algebra Jam, provides a set of tools to help students overcome two of the most serious impediments to successful collaboration: establishing common ground and maintaining group focus. These tools include tethered and untethered modes of operation including discrepancy notification, a goal-oriented team blackboard, object-oriented chat with collabicons, reification of problem solving roles, and the optional inclusion of a tutor agent as a virtual team participant. The tutor agent not only offers help and feedback on problem solving actions but also accumulates evidence about individual and group problem solving performance in a Bayesian inference network. The system is envisioned as a testbed for developing theories of teaming.

One of the main factors limiting the use of path consistency algorithms in real life applications is their high space complexity. C. Han and C. Lee (1988) presented a path consistency algorithm, PC-4, with O(n/sup 3/a/sup 3/) space complexity, which makes it practicable only for small problems. The author presents a new path consistency algorithm, PC-5, which has an O(n/sup 3/a/sup 2/) space complexity while retaining the worst case time complexity of PC-4. Moreover, the new algorithm exhibits a much better average case time complexity. The new algorithm is based on the idea (due to C. Bessiere (1994)) that, at any time, only a minimal amount of support has to be found and recorded for a labeling to establish its viability; one has to look for at new support only if the current support is eliminated. The author also shows that PC-5 can be improved further to yield an algorithm, PC5++, with even better average case performance and the same space complexity.