AI-powered educational technologies have demonstrated measurable benefits for learners, but their design and evaluation have largely centered on K-12 contexts. As a result, many AI-supported learning systems remain poorly aligned with the needs, constraints, and goals of adult learners. To better understand how AI systems function in adult education, this paper examines the deployment of several AI learning technologies developed within a multidisciplinary, national research institute in the United States focused on adult learning and online education. Drawing on longitudinal deployment data, we conducted a reflexive thematic analysis to identify recurring challenges and design considerations across systems. These insights were synthesized into a set of 19 design guidelines intended to inform future AI-supported adult learning technologies. We demonstrate the utility of these guidelines through a heuristic evaluation of the deployed systems. Lastly, we present a guideline exploration tool that aids in the ideation of technologies by connecting the guidelines to stakeholder statements surfaced in the analysis process.

As artificial intelligence (AI) becomes increasingly integrated into daily life, higher education must move beyond code-centric instruction to foster holistic AI literacy. We present a novel pedagogical approach that integrates embodied, unplugged activities into a university-level Introduction to AI course. Inspired by the effectiveness of CS Unplugged in K-12 education, our physical, collaborative activities gave students a first-person perspective on AI decision-making. Through interactive games modeling Search Algorithms, Markov Decision Processes, Q-learning, and Hidden Markov Models, students built an intuition for complex AI concepts and more easily transitioned to mathematical formalizations and code implementations. We present four unplugged AI activities, describe how to bridge from unplugged activities to plugged coding tasks, reflect on implementation challenges, and propose refinements. We suggest that unplugged activities can effectively bridge conceptual reasoning and technical skill-building in university-level AI education.

Existing AI agents typically execute multi-step tasks autonomously and only allow user confirmation at the end. During execution, users have little control, making the confirm-at-end approach brittle: a single error can cascade and force a complete restart. Confirming every step avoids such failures, but imposes tedious overhead. Balancing excessive interruptions against costly rollbacks remains an open challenge. We address this problem by modeling confirmation as a minimum time scheduling problem. We conducted a formative study with eight participants, which revealed a recurring Confirmation-Diagnosis-Correction-Redo (CDCR) pattern in how users monitor errors. Based on this pattern, we developed a decision-theoretic model to determine time-efficient confirmation point placement. We then evaluated our approach using a within-subjects study where 48 participants monitored AI agents and repaired their mistakes while executing tasks. Results show that 81 percent of participants preferred our intermediate confirmation approach over the confirm-at-end approach used by existing systems, and task completion time was reduced by 13.54 percent.

As governments increasingly adopt digital tools, public service chatbots have emerged as a growing communication channel.This paper explores the design considerations and engagement opportunities of public service chatbots, using a 311 chatbot from a metropolitan city as a case study.Our qualitative study consisted of official survey data and 16 interviews examining stakeholder experiences and design preferences for the chatbot.We found two key areas of concern regarding these public chatbots: individual-level and community-level.At the individual level, citizens experience three key challenges: interpretation, transparency, and social contextualization.Moreover, the current chatbot design prioritizes the efficient completion of individual tasks but neglects the broader community perspective.It overlooks how individuals interact and discuss problems collectively within their communities.To address these concerns, we offer design opportunities for creating more intelligent, transparent, community-oriented chatbots that better engage individuals and their communities.

In this work, we designed and developed Dice Adventure, a turnbased multiplayer game where three characters work together to reach their individual goals and then a shared team goal to complete each level.Using Dice Adventure as the environment, we hosted a game competition with two tracks: agent and player.Participants could join one or both by submitting an agent they developed and/or signing up to play with the agents submitted by other developers.We collected competition game play data as part of a human-AI teaming pilot study to understand team behaviors, performance, and to test our systems.Insights from the competition also informed the design of a randomized controlled study for future experiments and competitions, aimed at exploring key human-AI teaming questions-such as how role assignments, team compositions, and team dynamics influence team performance.Our work introduces a novel gaming environment to support future research on human-AI teaming and offers preliminary insights into the design of such studies.

Inspired by the human ability to learn and organize knowledge into hierarchical taxonomies with prototypes, this paper addresses key limitations in current deep hierarchical clustering methods. Existing methods often tie the structure to the number of classes and underutilize the rich prototype information available at intermediate hierarchical levels. We introduce deep taxonomic networks, a novel deep latent variable approach designed to bridge these gaps. Our method optimizes a large latent taxonomic hierarchy, specifically a complete binary tree structured mixture-of-Gaussian prior within a variational inference framework, to automatically discover taxonomic structures and associated prototype clusters directly from unlabeled data without assuming true label sizes. We analytically show that optimizing the ELBO of our method encourages the discovery of hierarchical relationships among prototypes. Empirically, our learned models demonstrate strong hierarchical clustering performance, outperforming baselines across diverse image classification datasets using our novel evaluation mechanism that leverages prototype clusters discovered at all hierarchical levels. Qualitative results further reveal that deep taxonomic networks discover rich and interpretable hierarchical taxonomies, capturing both coarse-grained semantic categories and fine-grained visual distinctions.

Intelligent tutoring systems leverage AI models of expert learning and student knowledge to deliver personalized tutoring to students. While these intelligent tutors have demonstrated improved student learning outcomes, it is still unclear how teachers might integrate them into curriculum and course planning to support responsive pedagogy. In this paper, we conducted a design study with five teachers who have deployed Apprentice Tutors, an intelligent tutoring platform, in their classes. We characterized their challenges around analyzing student interaction data from intelligent tutoring systems and built VisTA (Visualizations for Tutor Analytics), a visual analytics system that shows detailed provenance data across multiple coordinated views. We evaluated VisTA with the same five teachers, and found that the visualizations helped them better interpret intelligent tutor data, gain insights into student problem-solving provenance, and decide on necessary follow-up actions – such as providing students with further support or reviewing skills in the classroom. Finally, we discuss potential extensions of VisTA into sequence query and detection, as well as the potential for the visualizations to be useful for encouraging self-directed learning in students.

Machine learning often requires millions of examples to produce static, black-box models. In contrast, interactive task learning (ITL) emphasizes incremental knowledge acquisition from limited instruction provided by humans in modalities such as natural language. However, ITL systems often suffer from brittle, error-prone language parsing, which limits their usability. Large language models (LLMs) are resistant to brittleness but are not interpretable and cannot learn incrementally. We present VAL, an ITL system with a new philosophy for LLM/symbolic integration. By using LLMs only for specific tasks—such as predicate and argument selection—within an algorithmic framework, VAL reaps the benefits of LLMs to support interactive learning of hierarchical task knowledge from natural language. Acquired knowledge is human interpretable and generalizes to support execution of novel tasks without additional training. We studied users’ interactions with VAL in a video game setting, finding that most users could successfully teach VAL using language they felt was natural.

Intelligent Tutoring Systems (ITSs) have shown great potential in delivering personalized and adaptive education, but their widespread adoption has been hindered by the need for specialized programming and design skills. Existing approaches overcome the programming limitations with no-code authoring through drag and drop, however they assume that educators possess the necessary skills to design effective and engaging tutor interfaces. To address this assumption we introduce generative AI capabilities to assist educators in creating tutor interfaces that meet their needs while adhering to design principles. Our approach leverages Large Language Models (LLMs) and prompt engineering to generate tutor layout and contents based on high-level requirements provided by educators as inputs. However, to allow them to actively participate in the design process, rather than relying entirely on AI-generated solutions, we allow generation both at the entire interface level and at the individual component level. The former provides educators with a complete interface that can be refined using direct manipulation, while the latter offers the ability to create specific elements to be added to the tutor interface. A small-scale comparison shows the potential of our approach to enhance the efficiency of tutor interface design. Moving forward, we raise critical questions for assisting educators with generative AI capabilities to create personalized, effective, and engaging tutors, ultimately enhancing their adoption.

Intelligent tutors have shown success in delivering a personalized and adaptive learning experience. However, there exist challenges regarding the granularity of knowledge in existing frameworks and the resulting instructions they can provide. To address these issues, we propose HTN-based tutors, a new intelligent tutoring framework that represents expert models using Hierarchical Task Networks (HTNs). Like other tutoring frameworks, it allows flexible encoding of different problem-solving strategies while providing the additional benefit of a hierarchical knowledge organization. We leverage the latter to create tutors that can adapt the granularity of their scaffolding. This organization also aligns well with the compositional nature of skills.

Point-of-Care Ultrasound (POCUS) refers to clinician-performed and interpreted ultrasonography at the patient's bedside. Interpreting these images requires a high level of expertise, which may not be available during emergencies. In this paper, we support POCUS by developing classifiers that can aid medical professionals by diagnosing whether or not a patient has pneumothorax. We decomposed the task into multiple steps, using YOLOv4 to extract relevant regions of the video and a 3D sparse coding model to represent video features. Given the difficulty in acquiring positive training videos, we trained a small-data classifier with a maximum of 15 positive and 32 negative examples. To counteract this limitation, we leveraged subject matter expert (SME) knowledge to limit the hypothesis space, thus reducing the cost of data collection. We present results using two lung ultrasound datasets and demonstrate that our model is capable of achieving performance on par with SMEs in pneumothorax identification. We then developed an iOS application that runs our full system in less than 4 seconds on an iPad Pro, and less than 8 seconds on an iPhone 13 Pro, labeling key regions in the lung sonogram to provide interpretable diagnoses.

While recent advances in Artificial Intelligence and Machine Learning have demonstrated the potential for AI systems to outperform human experts in many domains, including games, AI systems still generally lack the ability to team with humans on complex tasks. One of the barriers to addressing this challenge is a lack of shared task domains in which to do basic research to study Human-Machine Teaming strategies. In our work, we employ speculative game design to create asymmetric cooperative games that can serve as test beds to study human-machine teaming challenges. In this paper, we will describe our general approach and detail the current state of our development efforts.

Artificial intelligence systems such as AlphaGo, AlphaGo Zero and AlphaZero, have demonstrated their advantages and competency over human players. However, little research has explored the possibility of applying such algorithms for educational purposes, such as teaching people to play strategy games. To investigate this gap, we designed and developed a Gomoku tutor that can provide instant/delayed feedback to users. We trained an expert model for Gomoku from scratch by using an open-source AlphaZero implementation and embedded this model into our Gomoku tutoring system. We plan to use this tutor to investigate two main research questions: 1) Can Game AI models, which are inhuman in their expertise, provide guidance that improves human learning? 2) How do different types of Game AI derived feedback affect people's learning outcomes? In this paper, we outline our experimental plans to investigate these questions.

CubeSats are currently gaining significant traction in Earth Observation, with increasingly advanced instrumentation such as hyperspectral imaging. However, the challenges of bringing such instrumentation to CubeSats are great; the platform suffers from severe physical, operational and budgetary constraints. Adopting a holistic design methodology may hold the key to allowing science-grade Earth Observation to be achieved from a CubeSat. Presented here is CHAFF (CubeSat Hyperspectral Application For Farming), a low-cost hyperspectral imager prototype, capable of taking 1024 spectral bands between 460 nm - 820 nm. CHAFF has been constructed using commercial off-the-shelf optics, in order to produce a design commensurate with the typical resources of a university CubeSat mission. CHAFF has been calibrated at the National Physical Laboratory, in order to assess the performance of the COTS optics. An impressive spectral resolution of 3.46 nm at 546 nm has been achieved, and 74.95% of CHAFF's pixels exhibit a linearity deviation of < 2%.

In field spectroscopy applications the autonomous field spectroradiometer is configured to measure the down-welling irradiance and up-welling radiance, from this it is possible to calculate the reflectance properties of the surface or canopy and how it may change over time and with illumination geometry. This paper reviews the merits and limitations of autonomous systems with “bi-conical” and hemispherical-conical geometries, single and multi-spectrometer configurations and the options for improving the measurement uncertainty in future autonomous systems.

Calibration and validation are fundamental for obtaining quantitative information from Earth Observation (EO) sensor data. Recognising this and the impending launch of at least five sensors in the next five years, the International Spaceborne Imaging Spectroscopy Technical Committee instigated a calibration and validation initiative. A workshop was conducted recently as part of this initiative with the objective of establishing a good practice framework for radiometric and spectral calibration and validation in support of spaceborne imaging spectroscopy missions. This paper presents the outcomes and recommendations for future work arising from the workshop.

How best to assess the creativity of a large number of designed artifacts remains an open problem. The typical approach is to have a panel of experts answer likert questions about individual artifacts. This process typically requires a substantial amount of training to ensure the judges achieve an acceptable level of agreement. Consequently, the approach does not scale well as it is infeasible to have a panel of experts regularly evaluate the creativity of a large number of designs. The current work explores an alternative approach that uses both individual and pairwise judgements from novice crowd workers to support reliable and scalable assessment of creative designs. This approach, which we call TrueCreativity, can operate over a set of evaluations from a large number of judges and appropriately weights their evaluations based on their past reliability and agreement with other judges. We show that this approach produces results that strongly correlate with another measure of creativity.

As educational games have become a larger field of study, there has been a growing need for analytic methods that can be used to assess game design and inform iteration. While much previous work has focused on the measurement of student engagement or learning at a gross level, we argue that new methods are necessary for measuring the alignment of a game to its target learning goals at an appropriate level of detail to inform design decisions. We present a novel technique that we have employed to examine alignment in an open-ended educational game. The approach is based on examining how the game reacts to representative student solutions that do and do not obey target principles. We demonstrate this method using real student data and discuss how redesign might be informed by these techniques.

Accurately determining field-of-view has rarely been considered in field spectroscopy where specifications for fore optics used are generally limited and the influence of the spectroradiometer rarely considered. The issue can be compounded with full wavelength spectroradiometric systems which include multiple spectrometers. In these systems, the size and alignment of the viewing optics and technology adopted to transfer light from the fore optic to individual spectrometers may cause significant nonuniformity of spectral response across the area of measurement support, and this area may not align with that assumed from the specification that is supplied for the fore optic. When recording spectra from heterogeneous earth surface targets, it is important to have the area of measurement support accurately defined as individual reflecting surfaces may be present in varying proportions within this area, and these proportions need to be determined to relate spectral reflectance or spectral radiance to state variables or target classifications being considered. The area of measurement support and the spatial and spectral responsivity of an ASD Field Spec Pro FR spectroradiometer and a SVC GER 3700 spectroradiometer have been determined by measuring the directional response function (DRF) of each instrument. This research highlights several areas of concern and makes recommendations for the improvement of field spectroradiometers and field spectroscopy methodologies. These results are specific to the spectroradiometer/fore optic combinations investigated and at the measurement distances specified. Although similar characteristics can be expected for other instruments/fore optics of the same design, and at other measurement distances, the DRFs will vary from those reported here.

In field spectroscopy, estimation of the bi-directional reflectance from a target requires paired, near-simultaneous measurement of downwelling irradiance and upwelling target reflected radiance. The conventional approach to field reflectance measurement adopts a single spectroradiometer with single field of view and reflectance panel to take sequential reference (irradiance) and target (radiance) measurements. However, previous research has shown that significant uncertainties in such measurements are introduced with fluctuations in ambient lighting, poorly defined fields-of-view, and known anisotropies of reference reflectance panels. We present a new field spectroradiometer with novel dual field-of-view input optics for repetitive autonomous reflectance measurement across the V-SWIR spectral range (400 to 1700 nm). Attention to the optical design has minimised uncertainties in field reflectance measurement by ensuring a uniform and well defined field-of-view and high accuracy cosine corrected irradiance fore-optic across the full spectral range of the measurement. The system includes just one set of silicon and InGaAs detector array based spectrometers to provide the dual field-of-view operation, saving on cost, weight and power consumption. An internal microprocessor provides full stand alone control and automation.

Accurately determining the field-of-view (FOV) is a basic requirement in photogrammetry and imaging spectroscopy but has rarely been considered in detail in field spectroscopy where the specifications for different spectroradiometers generally lack clarity or detail. The issue can be further compounded with full spectral systems (0.4 to 2.5 mum) which include multiple spectrometers; in these systems the size and alignment of the viewing optics may cause significant spectral non-uniformity across the theoretical measurement area. When recording spectra from heterogeneous targets it is important to have the FOV accurately defined because distinct reflecting surfaces may be present in varying proportions. We assessed the FOVs of the analytical spectral devices field spec pro FR and spectra Vista Corporation GER 3700 spectroradiometers. The resulting directional response functions are plotted, highlighting several areas of concern.