Social commonsense contains many human biases due to social and cultural influence (Sap et al., 2020;Emelin et al., 2020).We focus on identifying cultural biases in data, specifically causal assumptions and commonsense implications, that strongly influence model decisions for a variety of tasks designed for social impact.This enables us to examine data for bias by making explicit the causal (if-then, inferential) relations in social commonsense knowledge used for decision making, furthering interpretable commonsense reasoning from a dataset perspective.We apply our methods on 2 social tasks: emotion detection and perceived value detection.We identify influential social commonsense knowledge to explain model behavior in the following ways.First, we augment large-scale language models with social knowledge and show improvements for the tasks, indicating the implicit assumptions a model requires to be successful on each dataset.Second, we identify influential events in the datasets by using social knowledge to cluster data and demonstrate the influence that these events have on model behavior via leave-K-out experiments.This allows us to gain a datasetlevel understanding of the events and causal commonsense relationships that strongly influence predictions.We then analyze these relationships to detect influential cultural bias in each dataset.Finally, we use our influential event identification for detecting mislabeled examples and improve training and performance through their removal.We support our findings with manual analysis.

NLU models power several user facing experiences such as conversations agents and chat bots. Building NLU models typically consist of 3 stages: a) building or finetuning a pre-trained model b) distilling or fine-tuning the pre-trained model to build task specific models and, c) deploying the task-specific model to production. In this presentation, we will identify fairness considerations that can be incorporated in the aforementioned three stages in the life-cycle of NLU model building: (i) selection/building of a large scale language model, (ii) distillation/fine-tuning the large model into task specific model and, (iii) deployment of the task specific model. We will present select metrics that can be used to quantify fairness in NLU models and fairness enhancement techniques that can be deployed in each of these stages. Finally, we will share some recommendations to successfully implement fairness considerations when building an industrial scale NLU system.

Integrating external knowledge into commonsense reasoning tasks has shown progress in resolving some, but not all, knowledge gaps in these tasks.For knowledge integration to yield peak performance, it is critical to select a knowledge graph (KG) that is well-aligned with the given task's objective.We present an approach to assess how well a candidate KG can correctly identify and accurately fill in gaps of reasoning for a task, which we call KG-to-task match.We show this KGto-task match in 3 phases: knowledge-task identification, knowledge-task alignment, and knowledge-task integration.We also analyze our transformer-based KG-to-task models via commonsense probes to measure how much knowledge is captured in these models before and after KG integration.Empirically, we investigate KG matches for the SocialIQA (SIQA) (Sap et al., 2019b), Physical IQA (PIQA) (Bisk et al., 2020), and MCScript2.0(Ostermann et al., 2019) datasets with 3 diverse KGs: ATOMIC (Sap et al., 2019a), Con-ceptNet (Speer et al., 2017), and an automatically constructed instructional KG based on WikiHow (Koupaee and Wang, 2018).With our methods we are able to demonstrate that ATOMIC, an event-inference focused KG, is the best match for SIQA and MCScript2.0,and that the taxonomic ConceptNet and WikiHowbased KGs are the best matches for PIQA across all 3 analysis phases.We verify our methods and findings with human evaluation. 1Triple:Context: Tracy brought the kids to their dentist appointment, but it was scheduled during the school day.

Recent commonsense-reasoning tasks are typically discriminative in nature, where a model answers a multiple-choice question for a certain context.Discriminative tasks are limiting because they fail to adequately evaluate the model's ability to reason and explain predictions with underlying commonsense knowledge.They also allow such models to use reasoning shortcuts and not be "right for the right reasons".In this work, we present EX-PLAGRAPHS, a new generative and structured commonsense-reasoning task (and an associated dataset) of explanation graph generation for stance prediction.Specifically, given a belief and an argument, a model has to predict if the argument supports or counters the belief and also generate a commonsense-augmented graph that serves as non-trivial, complete, and unambiguous explanation for the predicted stance.We collect explanation graphs through a novel Create-Verify-And-Refine graph collection framework that improves the graph quality (up to 90%) via multiple rounds of verification and refinement.A significant 79% of our graphs contain external commonsense nodes with diverse structures and reasoning depths.Next, we propose a multi-level evaluation framework, consisting of automatic metrics and human evaluation, that check for the structural and semantic correctness of the generated graphs and their degree of match with ground-truth graphs.Finally, we present several structured, commonsense-augmented, and text generation models as strong starting points for this explanation graph generation task, and observe that there is a large gap with human performance, thereby encouraging future work for this new challenging task. 1

Reading comprehension QA tasks have seen a recent surge in popularity, yet most works have focused on fact-finding extractive QA.We instead focus on a more challenging multihop generative task (NarrativeQA), which requires the model to reason, gather, and synthesize disjoint pieces of information within the context to generate an answer.This type of multi-step reasoning also often requires understanding implicit relations, which humans resolve via external, background commonsense knowledge.We first present a strong generative baseline that uses a multi-attention mechanism to perform multiple hops of reasoning and a pointer-generator decoder to synthesize the answer.This model performs substantially better than previous generative models, and is competitive with current state-of-the-art span prediction models.We next introduce a novel system for selecting grounded multi-hop relational commonsense information from Con-ceptNet via a pointwise mutual information and term-frequency based scoring function.Finally, we effectively use this extracted commonsense information to fill in gaps of reasoning between context hops, using a selectivelygated attention mechanism.This boosts the model's performance significantly (also verified via human evaluation), establishing a new state-of-the-art for the task.We also show that our background knowledge enhancements are generalizable and improve performance on QAngaroo-WikiHop, another multi-hop reasoning dataset.