Sina Rismanchian, Yasaman Razeghi, Sameer Singh, Shayan Doroudi. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2025.

In 1971, social critic Ivan Illich published Deschooling Society, a controversial work that critiqued mainstream education systems and proposed a radical alternative. While his work remains controversial, re-examining his ideas might advance efforts to design for learning at scale. First, we examine three design principles that emerge from Illich's writing on learning webs: (1) a holistic perspective that incorporates multidisciplinary thinking (in Illich's case, blending philosophy, politics, sociology, economics, theology, and cybernetics), (2) learning webs as a framework for thinking about learning beyond the limitations of school, and (3) broadening our view of what should be scaled (e.g., scaling opportunities rather than content). Second, we discuss three tensions in Illich's work that relate to scaling learning: (1) decentralization vs. centralization, (2) place-based vs. online learning at scale, and (3) serving the advantaged vs. the disadvantaged. In discussing these ideas and tensions, we discuss contemporary technologies and models, which may be seen as similar to learning webs. Finally, we suggest that Illich's work offers an opportunity to further connect work that sits across two related research communities: Learning @ Scale and connected learning.

Access to undergraduate research is limited. One approach to broaden access is scaling up the mentee-to-mentor ratio (e.g., course-based undergraduate research experiences have a classroom of student mentees being led by one professor mentor). However, some mentors and mentees may prefer apprentice-style research, which is defined as research with a small mentee-to-mentor ratio. Pulling influences from non-scaled and scaled approaches, we implemented and evaluated the Community College to PhD (CC2PhD) Scholars Program, which was a community college research program. CC2PhD was designed to scale up non-personalized aspects of apprentice-style research while maintaining personalized one-on-one mentoring. The scaled non-personalized aspects of CC2PhD included the predefined mentoring curriculum and the research methods workshops. They were "scaled" in the sense that few people were involved in curriculum development and workshop instruction. These scaled resources can then be used by a large number of mentor-mentee pairs. We interviewed and surveyed seven mentor alumni and six mentee alumni to understand the effects of the scaled aspects of CC2PhD. We identified four themes: (1) improved time-related issues by saving time and facilitating time management, (2) influenced meeting content, (3) helped beginner mentors and mentees, and (4) increased mentors' willingness to volunteer. Future researchers can further scale-up and digitize our scaled research-apprentice model. For example, the mentoring curriculum and workshops can be adapted into a MOOC, which mentor-mentee pairs can reference from.

Given the growing interest in learnersourcing -- a pedagogically supported form of crowdsourcing that harnesses the knowledge and creativity of learners for the creation of learning resources -- we propose a theoretical framework to study, design, and deploy learnersourcing systems. By integrating ideas from crowdsourcing and learning theories focusing on learner-centered pedagogy, this work provides a review and classification of learnersourcing systems from the perspective of its three groups of stakeholders: (1) contributors, who are the learners who contribute new learning artifacts, (2) beneficiaries, who are the learners who learn from these artifacts, and (3) the instructional team who design and deploy learnersourcing tasks. The framework serves as a heuristic device for designing new learnersourcing systems and for considering the broader implications for learnersourcing in terms of workflow design, incentivizing contributors, quality-control, learning outcomes, delivering personalized learning experiences, ethical considerations, and its complementary relationship with AI in education.

In many online environments, such as massive open online courses and crowdsourcing platforms, many people solve similar complex tasks. As a byproduct of solving these tasks, a pool of artifacts are created that may be able to help others perform better on similar tasks. In this paper, we explore whether work that is naturally done by crowdworkers can be used as examples to help future crowdworkers perform better on similar tasks. We explore this in the context of a product comparison review task, where workers must compare and contrast pairs of similar products. We first show that randomly presenting one or two peer-generated examples does not significantly improve performance on future tasks. In a second experiment, we show that presenting examples that are of sufficiently high quality leads to a statistically significant improvement in performance of future workers on a near transfer task. Moreover, our results suggest that even among high quality examples, there are differences in how effective the examples are, indicating that quality is not a perfect proxy for pedagogical value.

Adaptive educational technologies have the capacity to meet the needs of individual students in theory, but in some cases, the degree of personalization might be less than desired, which could lead to inequitable outcomes for students. In this paper, we use simulations to demonstrate that while knowledge tracing algorithms are substantially more equitable than giving all students the same amount of practice, such algorithms can still be inequitable when they rely on inaccurate models. This can arise as a result of two factors: (1) using student models that are fit to aggregate populations of students, and (2) using student models that make incorrect assumptions about student learning. In particular, we demonstrate that both the Bayesian knowledge tracing algorithm and the N-Consecutive Correct Responses heuristic are susceptible to these concerns, but that knowledge tracing with the additive factor model may be more equitable. The broader message of this paper is that when designing learning analytics algorithms, we need to explicitly consider whether the algorithms act fairly with respect to different populations of students, and if not, how we can make our algorithms more equitable.

We consider the problem of off-policy policy selection in reinforcement learning: using historical data generated from running one policy to compare two or more policies. We show that approaches based on importance sampling can be unfair---they can select the worse of two policies more often than not. We then give an example that shows importance sampling is systematically unfair in a practically relevant setting; namely, we show that it unreasonably favors shorter trajectory lengths. We then present sufficient conditions to theoretically guarantee fairness. Finally, we provide a practical importance sampling-based estimator to help mitigate the unfairness due to varying trajectory lengths.

The gold standard for identifying more effective pedagogical approaches is to perform an experiment. Unfortunately, frequently a hypothesized alternate way of teaching does not yield an improved effect. Given the expense and logistics of each experiment, and the enormous space of potential ways to improve teaching, it would be highly preferable if it were possible to estimate in advance of running a study whether an alternative teaching strategy would improve learning. This is true even in learning at scale situations, since even if it is logistically easier to recruit a large number of subjects, it remains a high stakes environment because the experiment is impacting many real students. For certain classes of alternate teaching approaches, such as new ways to sequence existing material, it is possible to build student models that can be used as simulators to estimate the performance of learners under new proposed teaching methods. However, existing methods for doing so can overestimate the performance of new teaching methods. We instead propose the Robust Evaluation Matrix (REM) method which explicitly considers model mismatch between the student model used to derive the teaching strategy and that used as a simulator to evaluate the teaching strategy effectiveness. We then present two case studies from a fractions intelligent tutoring system and from a concept learning task from prior work that show how REM could be used both to detect when a new instructional policy may not be effective on actual students and to detect when it may be effective in improving student learning.

Many interesting real world domains involve reinforcement learning (RL) in partially observable environments. Efficient learning in such domains is important, but existing sample complexity bounds for partially observable RL are at least exponential in the episode length. We give, to our knowledge, the first partially observable RL algorithm with a polynomial bound on the number of episodes on which the algorithm may not achieve near-optimal performance. Our algorithm is suitable for an important class of episodic POMDPs. Our approach builds on recent advances in the method of moments for latent variable model estimation.

We explore how crowdworkers can be trained to tackle complex crowdsourcing tasks. We are particularly interested in training novice workers to perform well on solving tasks in situations where the space of strategies is large and workers need to discover and try different strategies to be successful. In a first experiment, we perform a comparison of five different training strategies. For complex web search challenges, we show that providing expert examples is an effective form of training, surpassing other forms of training in nearly all measures of interest. However, such training relies on access to domain expertise, which may be expensive or lacking. Therefore, in a second experiment we study the feasibility of training workers in the absence of domain expertise. We show that having workers validate the work of their peer workers can be even more effective than having them review expert examples if we only present solutions filtered by a threshold length. The results suggest that crowdsourced solutions of peer workers may be harnessed in an automated training pipeline.