Offline model selection (OMS), that is, choosing the best policy from a set of many policies given only logged data, is crucial for applying offline RL in real-world settings. One idea that has been extensively explored is to select policies based on the mean squared Bellman error (MSBE) of the associated Q-functions. However, previous work has struggled to obtain adequate OMS performance with Bellman errors, leading many researchers to abandon the idea. To this end, we elucidate why previous work has seen pessimistic results with Bellman errors and identify conditions under which OMS algorithms based on Bellman errors will perform well. Moreover, we develop a new estimator of the MSBE that is more accurate than prior methods. Our estimator obtains impressive OMS performance on diverse discrete control tasks, including Atari games.

There is much interest in predicting student performance in computer programming courses early in the semester to identify weak students who might benefit from targeted support. To this end, we analyzed detailed keystroke transcripts and outputs of compilation attempts during programming activities, both in and out of class. In linear regression models predicting grades, we identified multiple behavioral indicators and performance indicators that explained a significant portion of the variation in final grades using only the data collected within the first three weeks. Because the indicators identify specific behaviors and are generated automatically, they may be used as the basis for interventions instructors may use when counseling weaker students concerning their performance early in the course before they fall too far behind. Furthermore, in contrast with some other automated struggling-student detection models, our predictors are based on generic behaviors and generic performance metrics that can be extended to a wide range of introductory programming contexts. Our models also predict performance on a continuous scale rather than a binary "weak"/"not weak" classification, which would allow instructors to offer interventions to marginal students who want to improve, or to promising students who want to excel.