Developers and academics have grown increasingly interested in developing “trustworthy” artificial intelligence (AI). However, this aim is difficult to achieve in practice, especially given trust and trustworthiness are complex, multifaceted concepts that cannot be completely guaranteed nor built entirely into an AI system. We have drawn on the breadth of trust-related literature across multiple disciplines and fields to synthesize knowledge pertaining to interpersonal trust, trust in automation, and risk and trust. Based on this review we have (re)conceptualized trustworthiness in practice as being both (a) perceptual, meaning that a user assesses whether, when, and to what extent AI model output is trustworthy, even if it has been developed in adherence to AI trustworthiness standards, and (b) context-dependent, meaning that a user's perceived trustworthiness and use of an AI model can vary based on the specifics of their situation (e.g., time-pressures for decision-making, high-stakes decisions). We provide our reconceptualization to nuance how trustworthiness is thought about, studied, and evaluated by the AI community in ways that are more aligned with past theoretical research.

We describe the construction and assessment of a plan to foster student success in Computer Science (CS) in response to continued enrollment growth. We examined cross correlations of grades from student transcripts from the past four years to determine what patterns of grades in early classes were indicative of future success. The resulting statistics and visualizations showed that students generally did better in the first class than subsequent classes, suggesting that C grades in the first class might indicate difficulty. We also examined students enrolled in the pre-capstone software engineering course. The analysis showed that no graduating CS major had received a C in both first two programming courses. We examined the data to be sure that the proposal would not diminish the participation of members of underrepresented groups disproportionately. As a result, the CS faculty approved a change in prerequisite for the third course to require that student receive an A or a B in at least one of the first two courses. GPA data was also analyzed and found to be a poor predictor of success in computer science, demonstrating that our previous approach to limiting enrollment was inferior. This new approach has advantages over the previous GPA-based plan, but enrollment management plans risk making college harder for at risk students to succeed in CS. The biggest limitation of this plan is that it is expected to make only a small change in CS enrollment.

Hail causes billions of dollars in losses by damaging buildings, vehicles, and crops. Improving the spatial and temporal accuracy of hail forecasts would allow people to mitigate hail damage. We have developed an approach to forecasting hail that identifies potential hail storms in storm-scale numerical weather prediction models and matches them with observed hailstorms. Machine learning models, including random forests, gradient boosting trees, and linear regression, are used to predict the expected hail size from each forecast storm. The individual hail size forecasts are merged with a spatial neighborhood ensemble probability technique to produce a consensus probability of hail at least 25.4 mm in diameter. The system was evaluated during the 2014 National Oceanic and Atmospheric Administration Hazardous Weather Testbed Experimental Forecast Program and compared with a physics-based hail size model. The machine-learning-based technique shows advantages in producing smaller size errors and more reliable probability forecasts. The machine learning approaches correctly predicted the location and extent of a significant hail event in eastern Nebraska and a marginal severe hail event in Colorado.

Severe weather, including tornadoes, thunderstorms, wind, and hail annually cause significant loss of life and property. We are developing spatiotemporal machine learning techniques that will enable meteorologists to improve the prediction of these events by improving their understanding of the fundamental causes of the phenomena and by building skillful empirical predictive models. In this paper, we present significant enhancements of our Spatiotemporal Relational Probability Trees that enable autonomous discovery of spatiotemporal relationships as well as learning with arbitrary shapes. We focus our evaluation on two real-world case studies using our technique: predicting tornadoes in Oklahoma and predicting aircraft turbulence in the United States. We also discuss how to evaluate success for a machine learning algorithm in the severe weather domain, which will enable new methods such as ours to transfer from research to operations, provide a set of lessons learned for embedded machine learning applications, and discuss how to field our technique.

Inspired by research that indicates that direct competition is not always comfortable for female students, we redesigned an existing class competition to permit students to choose whether they wished to participate in either direct or indirect competition. We pilot tested it in the Spring of 2013 in a undergraduate/graduate class on introductory artificial intelligence at the University of Oklahoma. Although the results for female students are inconclusive due to their small number, we observed that international students embraced the indirect competitions. This suggests that allowing the option of indirect competition may also appeal other groups of students who can be marginalized in engineering. Our results indicate the international students prefer the less risky option of indirect competition.

Many real world domains are inherently spatiotemporal in nature. In this work, we introduce significant enhancements to two spatiotemporal relational learning methods, the spatiotemporal relational probability tree and the spatiotemporal relational random forest, that increase their ability to learn using spatiotemporal data. We enabled the models to formulate questions on both objects and the scalar and vector fields within and around objects, allowing the models to differentiate based on the gradient, divergence, and curl and to recognize the shape of point clouds defined by fields. This enables the model to ask questions about the change of a shape over time or about its orientation. These additions are validated on several real-world hazardous weather datasets. We demonstrate that these additions enable the models to learn robust classifiers that outperform the versions without these new additions. In addition, analysis of the learned models shows that the findings are consistent with current meteorological theories.

We introduce spatiotemporal relational probability trees (SRPTs), probability estimation trees for relational data that can vary in both space and time. The SRPT algorithm addresses the exponential increase in search complexity through sampling. We validate the SRPT using a simulated data set and we empirically demonstrate the SRPT algorithm on two real-world data sets.

We introduced an arcade-style gaming environment for use in a mixed undergraduate and graduate introductory artificial intelligence (AI) course. Our primary goal in this course was to provide students with a "significant learning experience" [3]. We achieved this goal by creating projects based in the game environment that illustrate several major AI topic areas. These projects were designed to be challenging, enjoyable, and to demonstrate AI programming in a realistic environment. Each of the projects was designed to be feasible for all the students yet flexible enough to allow the stronger students to explore alternative solutions. We evaluated our success in achieving these goals through student evaluations, comments, and exam grades.