The rapid proliferation of Generative AI (GenAI) into diverse, high-stakes domains necessitates robust and reproducible evaluation methods. However, practitioners often resort to ad-hoc, non-standardized scripts, as common metrics are often unsuitable for specialized, structured outputs (e.g., automated plans, time-series) or holistic comparison across modalities (e.g., text, audio, and image). This fragmentation hinders comparability and slows AI system development. To address this challenge, we present GAICo (Generative AI Comparator): a deployed, open-source Python library that streamlines and standardizes GenAI output comparison. GAICo provides a unified, extensible framework supporting a comprehensive suite of reference-based metrics for unstructured text, specialized structured data formats, and multimedia (images, audio). Its architecture features a high-level API for rapid, end-to-end analysis, from multi-model comparison to visualization and reporting, alongside direct metric access for granular control. We demonstrate GAICo's utility through a detailed case study evaluating and debugging complex, multi-modal AI Travel Assistant pipelines. GAICo empowers AI researchers and developers to efficiently assess system performance, make evaluation reproducible, improve development velocity, and ultimately build more trustworthy AI systems, aligning with the goal of moving faster and safer in AI deployment. Since its release on PyPI in Jun 2025, the tool has been downloaded over 16K times, across versions, by Dec 2025, demonstrating growing community interest.

The rapid evolution of Generative AI, yielding outputs across text, structured data, images, and audio, has outpaced the development of standardized evaluation tools, leading to fragmented and non-reproducible practices. GAICo (Generative AI Comparator) offers a solution: a deployed, open-source Python library that provides a unified, extensible, and reproducible framework for multi-modal GenAI evaluation. Our demonstration highlights GAICo’s utility through a practical case study: evaluating and debugging composite AI Travel Assistant pipelines. We show how GAICo facilitates isolating performance issues, for instance, distinguishing orchestrator LLM planning deficiencies from specialist image model generation flaws, by consistently comparing diverse outputs against tailored references. This framework streamlines development, improves system reliability, and promotes reproducible evaluation, making it a critical tool for building safer and more effective AI. Its rapid adoption, evidenced by over 16,000 downloads in the first 6 months, underscores its relevance and impact within the AI community.

AI models are increasingly accessible through chatbots and other applications, but their black-box nature and sensitivity to small changes in the input make them hard to interpret and trust. Existing correlation-based robustness metrics fail to explain model errors or isolate causal effects. To address this, I propose ARC (AI Rating through Causality), a causally-grounded framework for rating AI models based on their robustness. ARC evaluates robustness by quantifying statistical and confounding biases, as well as the impact of perturbations on model performance across diverse tasks. ARC produces interpretable raw scores and ratings, helping developers and users make informed decisions about model robustness. Two future directions include: (1) deriving raw scores for composite models from their component scores, and (2) combining ratings with traditional explainable AI approaches to provide a more holistic view of model behavior.

Modern artificial intelligence (AI) methods have been used to solve problems that many humans struggle to solve. This opens up new opportunities for knowledge discovery and education. We demonstrate ALLURE, an educational AI system for learning to solve the Rubik’s cube that is designed to help students improve their problem solving skills. ALLURE can both find and explain its own strategies for solving the Rubik’s cube as well as build on user-provided strategies. Collaboration between AI and user happens using visual and natural language modalities.

Artificial Intelligence (AI) systems like facial recognition systems and sentiment analyzers are known to exhibit model uncertainty which can be perceived as algorithmic bias in most cases. The aim of my Ph.D. is to examine and control the bias present in these AI systems by establishing causal relationships and also assigning a rating to these systems, which helps the user to make an informed selection when choosing from different systems for their application.