Deep Learning models are often called ‘Black Box’ models because of the difficulty in providing logical, actionable explanations of their decisions. Attribution methods try to solve this issue by determining the contribution of each feature to the decision. Recently, the Integrated Gradients method has shown promising results in variable attribution for differentiable models [1]. Integrated Gradients belong to the class of a more general attribution method called Path Integrated Gradients. Path Integrated Gradients is a parametric path-dependent and baseline-dependent attribution method. If the path is a straight line, the method becomes the canonical Integrated Gradients method. Choosing any other path leads to different attributions in general. Similarly, using different baselines may also lead to a change in attributions. Sundararajan et al. [1] recommend choosing a baseline that contains as little signal (discriminatory features used by the model for solving the task at hand) as possible, with the expectation that a comparison between the baseline and any input will reveal the signals of that input. In this paper, we show that choosing baselines with signals can also have very informative, distinct, and useful interpretations. We also propose two useful applications in the credit industry using two such baselines.

Food deserts are geographic areas where people have limited access to healthy and affordable food. Millions of people in the US are experiencing adverse effects stemming from living in such areas and it becomes an important challenge for federal and local authorities to improve the quality of life of people living in food deserts. The majority of studies in this field explicitly consider income level when analyzing problems of food deserts, thus shadowing other attributes that might provide deeper insights into the problems of the food desert’s population. The current paper assesses the significance of subtler, non-income related characteristics of census tracts by using a new variable attribution technique Appley which approximates Shapley values in linear time. The results show that attributes of census tracts that fall into the set of the so called "social determinants of health" have similar classification power compared to models that explicitly use income variables. The results suggest that each food desert has its own unique set of problems with social determinants of health with varying levels of intensities of each problem and this allows to clearly distinguish not only food deserts from non-food desert areas but also distinguish food deserts from each other. Appley attributes localized importance scores to food deserts’ social determinants of health, thus showing which issues are more severe in a given food desert compared to another one. This allows for devising food desert-specific solutions to the health issues of desert residents thus letting authorities improve the quality of life of people in a targeted and budget-efficient manner.

We have seen complex deep learning models out-performing human benchmarks in many areas (e.g. computer vision, natural language processing). Clever architectures and higher model complexity are two of the major drivers of such outstanding performances. Higher model complexity generally makes the decision-making process of a model opaque to human perception. But understanding the decision-making process is very important for many reasons including enhancing trust in the model’s prediction, improving model robustness, gaining actionable insight from why a model made a particular prediction and discovering new knowledge about a problem. Model explainability has been an active area of research for some time now, but the problem is still far from being solved. An established way of model explanation is to assign a score to each variable (also known as variable attribution), which represents the importance of the variable in a particular prediction of a model. In a lot of techniques, the scoring process involves distributing the output to each variable. This is challenging when the model is complex and consists of a high degree of interaction terms. A coalition game theoretic approach called Shapley Value provides a fair way to tackle the challenge. However, the growth of computation time of the exact Shapley Values is exponential in the number of variables. Hence, many attribution techniques use approximations instead of the exact Shapley Value as attribution. In this manuscript, we propose a novel variable attribution technique called Appley (short for Approximate Shapley) by approximating the Shapley Values in linear time. Moreover, we show that the “Appley” attributions are generally closer to the exact Shapley Values than an existing well-known and comparable attribution technique.

In modern society, epistemic uncertainty limits trust in financial relationships, necessitating transparency and accountability mechanisms for both consumers and lenders. One upshot is that credit risk assessments must be explainable to the consumer. In the United States regulatory milieu, this entails both the identification of key factors in a decision and the provision of consistent actions that would improve standing. The traditionally accepted approach to explainable credit risk modeling involves generating scores with Generalized Linear Models (GLMs) - usually logistic regression, calculating the contribution of each predictor to the total points lost from the theoretical maximum, and generating reason codes based on the 4 or 5 most impactful predictors. The industry standard approach is not directly applicable to a more expressive and flexible class of nonlinear models known as neural networks. This paper demonstrates that an eXplainable AI (XAI) variable attribution technique known as Integrated Gradients (IG) is a natural generalization of the industry standard to neural networks. We also discuss the unique semantics surrounding implementation details in this nonlinear context. While the primary purpose of this paper is to introduce IG to the credit industry and argue for its establishment as an industry standard, a secondary goal is to familiarize academia with the legislative constraints – including their historical and philosophical roots – and sketch the standard approach in the credit industry since there is a dearth of literature on the topic.

The coronavirus disease 2019 (COVID-19) caused a pandemic outbreak with affecting 213 nations worldwide. Global policymakers are imposing many measures to slow and reduce the rapid growth of the infections. On the other hand, the healthcare system is encountering significant challenges for a massive number of COVID-19 confirmed or suspected individuals seeking treatment. Therefore, estimating the number of confirmed cases is necessary to provide valuable insights into the growth of the outbreak and facilitate policy making process. In this study, we apply ARIMA models as well as LSTM-based recurrent neural network to forecast the daily cumulative confirmed cases. The LSTM architecture generates more precise forecasting by leveraging both short- and long-term temporal dependencies from the pandemic time series data. Due to the stochastic nature in optimization and random initialization of weights in neural network, the LSTM based model produce less reproducible outcome. In this paper, we propose a reproducible-LSTM (r-LSTM) framework that produces a reproducible and robust results leveraging z-score outlier detection method. We performed five round of nested cross validation to show the consistency in evaluating model performance. The experimental results demonstrate that r-LSTM outperformed the ARIMA model producing minimum MAPE, RMSE, and MAE.