With the proliferation of user-friendly generative-AI tools, many government agencies, such as the National Science Foundation and the National Institutes of Health, have started exploring how these tools could improve one of their primary functions: managing grant programs to distribute funds for research. Government bans on the use of generative-AI for initial grant application evaluation limits the use of this technology. However, this research shows that a post hoc evaluative approach within the domain of Responsible AI provides a means by which government agencies can evaluate the big data collected through their grant programs. This proposed framework, Responsible AI for Evaluation (RAI-E), achieves this goal by curating big governmental administrative data using grant evaluation rubrics followed by model fine-tuning for agency-specific evaluation tasks. Convergence between model- and human-generated evaluations then enables robust diagnosis of grant program equity and fairness via Responsible AI and algorithm auditing methods. Preliminary experiments show improved model performance through the instruction-tuning of open source language models for evaluating government grant applications. Further research will explore the auditing of the underlying algorithms, devising a method for algorithm-in-the-loop frameworks to aid government administrators in evaluating their human-driven program performance.

This study compares two machine learning algorithms, decision trees and k-nearest neighbor, on the effectiveness of predicting success for organizations applying for arts and culture-related grants. Previous work in predictive modeling for grant applications found that accurate predictions were possible but were unable to provide explanations for model performance. Additionally, the data used in prior studies focused primarily on applicant academic pedigree and grant application history rather than details of grant application materials. Our research remedies this issue by training "white-box" models on application data from arts and cultural organizations applying to a government grant program in Chicago. Our results found that decision trees proved more accurate than k-nearest neighbors in predicting grant application success. Additionally, because the decision tree carries with it a ranking of the key factors for success, our approach provides useful information that may be beneficial to government agencies in improving the decision process and for applicants in evaluating past grant programs and predicting the success of future applications.