Scientists often use meta-analysis to characterize the impact of an intervention on some outcome of interest across a body of literature. However, threats to the utility and validity of meta-analytic estimates arise when scientists average over potentially important variations in context like different research designs. Uncertainty about quality and commensurability of evidence casts doubt on results from meta-analysis, yet existing software tools for meta-analysis do not provide an explicit software representation of these concerns. We present MetaExplorer, a prototype system for meta-analysis that we developed using iterative design with meta-analysis experts to provide a guided process for eliciting assessments of uncertainty and reasoning about how to incorporate them during statistical inference. Our qualitative evaluation of MetaExplorer with experienced meta-analysts shows that imposing a structured workflow both elevates the perceived importance of epistemic concerns and presents opportunities for tools to engage users in dialogue around goals and standards for evidence aggregation.

The development of methods that can generate compelling and accurate explanations of machine learning models and their predictions would mark a major advance in the state of the art by enabling developers and end users to detect model shortcomings, understand why predictions are made, and enable rich human-automation dialog. In our ongoing research, we seek to make contributions in three areas. First, our system utilizes abductive inference to produce more compelling and accurate explanations than prior methods. Second, our approach fully integrates explanations as actionable tools within a recommendation system. Third, accumulated feedback and abductive reasoning support the discovery of new features that hold the potential for improving subsequent rounds of machine learning.

One of the most critical problems facing the information security community is the threat of a malicious insider abusing his computer privileges to modify, remove, or prevent access to an organization's data. An insider is considered trusted (at least implicitly) by his organization because he is granted access to its computing environment. Whether or not that insider is in fact trustworthy is a question that lies at the heart of the insider threat problem. Complicating this problem is the fact that there is no "one size fits all" description of a malicious insider. Motivations, objectives, cyber expertise, system privileges all can and do vary from one case to the next.

Computer and network systems are vulnerable to attacks. Abandoning the existing huge infrastructure of possibly-insecure computer and network systems is impossible, and replacing them by totally secure systems may not be feasible or cost effective. A common element in many attacks is that a single user will often attempt to intrude upon multiple resources throughout a network. Detecting the attack can become significantly easier by compiling and integrating evidence of such intrusion attempts across the network rather than attempting to assess the situation from the vantage point of only a single host. To solve this problem, we suggest an approach for distributed recognition and accountability (DRA), which consists of algorithms which “process”, at a central location, distributed and asynchronous “reports” generated by computers (or a subset thereof) throughout the network. Our highest-priority objectives are to observe ways by which an individual moves around in a network of computers, including changing user names to possibly hide his/her true identity, and to associate all activities of multiple instances of the same individual to the same networkwide user. We present the DRA algorithm and a sketch of its proof under an initial set of simplifying albeit realistic assumptions. Later, we relax these assumptions to accommodate pragmatic aspects such as missing or delayed “reports”, clock skew, tampered “reports”, etc. We believe that such algorithms will have widespread applications in the future, particularly in intrusion-detection systems.