Research on Collaborative Problem Solving (CPS) has traditionally examined how humans rely on one another cognitively and socially to accomplish tasks together. With the rapid advancement of AI and large language models, however, a new question emerge: what happens to team dynamics when one of the “teammates" is not human? In this study, we investigate how the integration of an AI teammate – a fully autonomous GPT-4 agent with social, cognitive, and affective capabilities – shapes the socio-cognitive dynamics of CPS. We analyze discourse data collected from human-AI teaming (HAT) experiments conducted on a novel platform specifically designed for HAT research. Using two natural language processing (NLP) methods, specifically Linguistic Inquiry and Word Count (LIWC) and Group Communication Analysis (GCA), we found that AI teammates often assumed the role of dominant cognitive facilitators, guiding, planning, and driving group decision-making. However, they did so in a socially detached manner, frequently pushing agenda in a verbose and repetitive way. By contrast, humans working with AI used more language reflecting social processes, suggesting that they assumed more socially oriented roles. Our study highlights how learning analytics can provide critical insights into the socio-cognitive dynamics of human-AI collaboration.

Generative artificial intelligence (AI) has become a prominent presence in classrooms, yet relatively little is known about how students actually engage with such tools in authentic school contexts. This study examines more than 17,000 messages across 1,512 conversations from 484 middle school students using a classroom-based generative AI writing tutor. We extracted linguistic, cognitive, and interactional features, reduced dimensionality with principal component analysis, and applied clustering to identify conversation and student-level patterns of engagement. Results revealed five conversation profiles—ranging from directive to reflective dialogue—and four student profiles, including collaborators, transactional tool users, independent thinkers, and chatters. These patterns aligned with both pedagogical intent and individual orientation, underscoring that student–AI dialogue is heterogeneous but systematic. Findings contribute empirical evidence to debates about generative AI in education and provide a methodological framework for analyzing human–AI interaction in learning analytics

We investigated the feasibility of automating the modeling of collaborative problem-solving skills encompassing both social and cognitive aspects. Leveraging a diverse array of cutting-edge techniques, including machine learning, deep learning, and large language models, we embarked on the classification of qualitatively coded interactions within groups. These groups were composed of four undergraduate students, each randomly assigned to tackle a decision-making challenge. Our dataset comprises contributions from 514 participants distributed across 129 groups. Employing a suite of prominent machine learning methods such as Random Forest, Support Vector Machines, Naive Bayes, Recurrent and Convolutional Neural Networks, BERT, and GPT-2 language models, we undertook the intricate task of classifying peer interactions. Notably, we introduced a novel task-based train-test split methodology, allowing us to assess classification performance independently of task-related context. This research carries significant implications for the learning analytics field by demonstrating the potential for automated modeling of collaborative problem-solving skills, offering new avenues for understanding and enhancing group learning dynamics.

Promoting diversity and equity in STEM requires ongoing assessment of progress in addressing gender disparities. Historically, women have faced challenges such as a lower sense of belonging and reduced persistence in STEM higher education. Prior research found robust correlations between a sense of belonging and STEM persistence and suggests intervention focusing on cultivating women's belonging to increase their persistence interest. We ask whether these gender gaps persist amongst motivated STEM learners, and whether we find men and women motivated by the same reasons in a global online learning community. We found that while women reported lower average sense of belonging to their field of study, their sense of identity with STEM is stronger than men. Our findings suggest no gender differences in persistence intent, but two key factors: sense of belonging and field identity were significant predictors of STEM persistence intent, with stronger coefficients from sense of belonging observed for both men and women. Notably, self-efficacy was an additional predictor of STEM intent only for women, even though women reported lower self-efficacy than men. These findings allow us to understand the nuances of different contributing factors to learners' interest in pursuing STEM. With such understanding, it allows us to more effectively strategize how to leverage AI in a scaled learning environment to promote equitable learning. We discussed the implications for fostering an equitable learning environment and supporting an international STEM community.

Collaborative problem-solving (CPS) is one of the most essential 21st century skills for success across educational and professional settings. The hidden-profile paradigm is one of the most prominent avenues of studying group decision making and underlying issues in information sharing. Previous research on the hidden-profile paradigm has primarily focused on static constructs (e.g., group size, group expertise), or on the information itself (whether certain pieces of information is being shared). In the current study, we propose a lens on individual and group’s collaborative problem-solving skills, to explore the relationships between dynamic discourse processes and decision making in a distributed information environment. Specifically, we sought to examine CPS skills in association with decision change and productive decision-making. Our results suggest that while sharing information has significantly positive association with decision change and effective decision-making, other aspects of social processes appear to be negatively correlated with these outcomes. Cognitive CPS skills, however, exhibit a strong positive relationship with making a (productive) change in students final decisions. We also find that these results are more pronounced at the group level, particularly with cognitive CPS skills. Our study shed lights on a more nuanced picture of how social and cognitive CPS interactions are related to effective information sharing and decision making in collaborative problem-solving interactions.

Large-scale online learning environments present new opportunities to address the need for greater inclusivity in education. Unlike residential environments, which have physical and logistic constraints (e.g., classroom configurations, sizes, and scheduling) that impede our ability to enact more inclusive pedagogy, online learning environments can be personalized and adapted to individual learner needs. As these environments are completely technology mediated, they offer an almost infinite design space for innovation. Social-scientific research on inclusivity in residential settings provides insight into how we might design for online learning environments, however evidence of efficacious digital implementations of these insights is limited. This workshop aims to advance our understanding of the ways in which adaptivity can be leveraged to buttress inclusivity in STEM learning. Through brief paper presentations and collaborative activities we intend to outline design opportunities in the scaled learning space for creating more inclusive environments.

There has been limited research on how perceptions of socioeconomic status (SES) and opinion difference could influence peer feedback in Massive Open Online Courses (MOOCs). Using social comparison theory [12], we investigated the influence of ability and opinion-related factors on peer feedback text in a data science MOOC. Perceived SES of peers and the formality of written responses were used as the ability-related factor, while agreement between learners represented the opinion-related factor. We focused on understanding the behaviors of those learners who are most prevalent in MOOCs; those from high socioeconomic countries. Through two studies, we found a strong and repeated influence of agreement on affect and formality in feedback to peers. While a mediation effect of perceived SES was found, a significant effect of formality was not. This work contributes to an understanding of how social comparison theory can be operationalized in online peer writing environments.

There has been long-standing stereotypes on men and women's communication styles, such as men using more assertive or aggressive language and women showing more agreeableness and emotions in interactions. In the context of collaborative learning, male learners often believed to be more active participants while female learners are less engaged. To further explore gender differences in learners communication behavior and whether it has changed in the context of online synchronous collaboration, we examined students interactions at a sociocognitive level with a methodology called Group Communication Analysis (GCA). We found that there were no significant differences between men and women in the degree of participation. However, women exhibited significantly higher average social impact, responsivity and internal cohesion compared to men. We also compared the proportion of learners interaction profiles, and results suggest that women are more likely to be effective and cohesive communicators. We discussed implications of these findings for pedagogical practices to promote inclusivity and equity in collaborative learning online.

MOOCs have developed into multiple learning design models with a wide range of objectives. Teach-Outs are one such example, aiming to drive meaningful discussions around topics of pressing social urgency without the use of formal assessments. Given this approach, it is crucial to evaluate learners' engagement in the discussion forum to understand their experiences. This paper presents a pilot study that applied unsupervised natural language processing techniques to understand what and how students engage in dialogue in a Teach-Out. We used topic modeling to discover the emerging topics in the discussion forums and evaluated the on-topicness of the discussions (i.e. the degree to which discussions were relevant to the Teach-Out content). We also applied content analysis to investigate the sentiments associated with the discussions. We have taken a step toward extracting structure from students' discussions to understand learning behaviors happen in the discussion forum. This is the first study to analyze discussion forums in a Teach-Out.

There has been a growing trend in higher education towards increased use and adoption of Massive Open Online Courses (MOOCs). Despite this interest in learning at scale, limited work has compared MOOC activity across subsequent course offerings. In this study, we explore forum activity in ten iterations of the same MOOC. Our results suggest that participation in MOOC forums has changed over the past four years of delivery. First, overall participation in MOOC forums have decreased. Second, in later iterations cohorts of more committed forum users start to resemble formal online courses in size (67>n>36). However, despite the smaller groups of learners that should find it easier to form connections with one another, our analysis did not reveal the expected increase in the quality of social activity. Instead, MOOC forums evolved into smaller on-task question and answer (Q&A) spaces, not capitalizing on the opportunities for social learning. We discuss practical and research implications of such changes.

The facilitation of interpersonal relationships within a respectful learning climate is an important aspect of teaching practice. However, in large-scale online contexts, such as MOOCs, the number of learners and highly asynchronous nature militates against the development of a sense of belonging and dyadic trust. Given these challenges, instead of conventional instruments that reflect learners' affective perceptions, we suggest a set of indicators that can be used to evaluate social activity in relation to the participation structure. These group-level indicators can then help teachers to gain insights into the evolution of social activity shaped by their facilitation choices. For this study, group-level indicators were derived from measuring information exchange activity between the returning MOOC posters. By conceptualizing this group as an identity-based community, we can apply exponential random graph modelling to explain the network's structure through the configurations of direct reciprocity, triadic-level exchange, and the effect of participants demonstrating super-posting behavior. The findings provide novel insights into network amplification, and highlight the differences between the courses with different facilitation strategies. Direct reciprocation was characteristic of non-facilitated groups. Exchange at the level of triads was more prominent in highly facilitated online communities with instructor's involvement. Super-posting activity was less pronounced in networks with higher triadic exchange, and more pronounced in networks with higher direct reciprocity.

There is an emerging trend in higher education for the adoption of massive open online courses (MOOCs). However, despite this interest in learning at scale, there has been limited work investigating how MOOC participants have changed over time. In this study, we explore the temporal changes in MOOC learners' language and discourse characteristics. In particular, we demonstrate that there is a clear trend within a course for language in discussion forums to be of both more on-topic and reflective of deep learning in subsequent offerings of a course. We measure this in two ways, and demonstrate this trend through several repeated analyses of different courses in different domains. While not all courses show an increase beyond statistical significance, the majority do, providing evidence that MOOC learner populations are changing as the educational phenomena matures.

Connections established between learners via interactions are seen as fundamental for connectivist pedagogy. Connections can also be viewed as learning outcomes, i.e. learners' social capital accumulated through distributed learning environments. We applied linear mixed effects modeling to investigate whether the social capital accumulation interpreted through learners' centrality to course interaction networks, is influenced by the language learners use to express and communicate in two connectivist MOOCs. Interactions were distributed across the three social media, namely Twitter, blog and Facebook. Results showed that learners in a cMOOC connect easier with the individuals who use a more informal, narrative style, but still maintain a deeper cohesive structure to their communication.

The present study investigated unimodal and multimodal emotion perception by humans, with an eye for applying the findings towards automated affect detection. The focus was on assessing the reliability by which untrained human observers could detect naturalistic expressions of non-basic affective states (boredom, engagement/flow, confusion, frustration, and neutral) from previously recorded videos of learners interacting with a computer tutor. The experiment manipulated three modalities to produce seven conditions: face, speech, context, face+speech, face+context, speech+context, face+speech+context. Agreement between two observers (OO) and between an observer and a learner (LO) were computed and analyzed with mixed-effects logistic regression models. The results indicated that agreement was generally low (kappas ranged from .030 to .183), but, with one exception, was greater than chance. Comparisons of overall agreement (across affective states) between the unimodal and multimodal conditions supported redundancy effects between modalities, but there were superadditive, additive, redundant, and inhibitory effects when affective states were individually considered. There was both convergence and divergence of patterns in the OO and LO data sets; however, LO models yielded lower agreement but higher multimodal effects compared to OO models. Implications of the findings for automated affect detection are discussed.

We explored the possibility of predicting learners' affective states (boredom, flow/engagement, confusion, and frustration) by monitoring variations in the cohesiveness of tutorial dialogues during interactions with AutoTutor, an intelligent tutoring system with conversational dialogues. Multiple measures of cohesion (e.g., pronouns, connectives, semantic overlap, causal cohesion, coreference) were automatically computed using the Coh-Metrix facility for analyzing discourse and language characteristics of text. Cohesion measures in multiple regression models predicted the proportional occurrence of each affective state, yielding medium to large effect sizes. The incidence of negations, pronoun referential cohesion, causal cohesion, and co-reference cohesion were the most diagnostic predictors of the affective states. We discuss the generalizability of our findings to other domains and tutoring systems, as well as the possibility of constructing real-time, cohesion-based affect detectors.