Computerized Adaptive Testing (CAT) offers an efficient and personalized method for assessing examinee proficiency by dynamically adjusting test questions based on individual performance. Compared to traditional, non-personalized testing methods, CAT requires fewer questions and provides more accurate assessments. As a result, CAT has been widely adopted across various fields, including education, healthcare, sports, sociology, and the evaluation of AI models. While traditional methods rely on psychometrics and statistics, the increasing complexity of large-scale testing has spurred the integration of machine learning techniques. This paper aims to provide a machine learning-focused survey on CAT, presenting a fresh perspective on this adaptive testing paradigm. We delve into measurement models, question selection algorithm, bank construction, and test control within CAT, exploring how machine learning can optimize these components. Through an analysis of current methods, strengths, limitations, and challenges, we strive to develop robust, fair, and efficient CAT systems. By bridging psychometric-driven CAT research with machine learning, this survey advocates for a more inclusive and interdisciplinary approach to the future of adaptive testing.

Despite the proven efficacy of HPV vaccines, uptake remains limited in many regions, including China. This study investigates how health beliefs and emotional responses evolve across text-, audio-, and video-based platforms by analyzing data from three representative platforms in China, including 273,357 posts from Weibo (text-based), 1,228 podcasts from Ximalaya (audio-based), and 1,225 videos from Douyin (video-based) from July 2018 to March 2023. The comparisons are conducted under four dimensions as suggested by the Health Belief Model (HBM), including susceptibility, severity, benefits, and barriers. Our findings reveal distinct modality-specific patterns. For instance, a text-based platform tends to amplify barriers and negativity, an audio-based platform enables balanced and sustained discussions, and a video-based platform highlights personal anecdotes and drives rapid sentiment shifts. By highlighting these modality-specific differences and addressing potential cross-modal incongruities at the content level, we provide actionable insights for public health communicators, policymakers, and platform designers to tailor strategies, foster informed decision-making, and ultimately enhance HPV vaccine uptake in complex social media ecosystems.

In recent years, live streaming develops rapidly, attracting an increasing number of users. Providing personalized live streaming recommendations is crucial for platform improvement in enhancing user experience and increasing profitability. In live streaming scenarios, users often enter the same live streaming room multiple times, send chat messages and give virtual gifts repeatedly. However, existing recommendation models fail to effectively model the complex multiple types of repeated behaviors of users in live streaming scenarios, thus failing to achieve optimal recommendation results. To address this issue, we propose a novel live recommendation model called MRB4LS based on multiple types of repeated behaviors data. Specifically, we first construct three bipartite graphs to better capture the effects of multiple types of users’ repeated behaviors, including enter, chat, and gift. Second, we introduce a graph attention network named RepGAT, which explicitly learns from users’ repeated behaviors. RepGAT incorporates the number of repeated interactions between nodes when computing normalized attention coefficients, enabling a deeper exploration of users’ preferences and the heterogeneous strength of the interaction relationship between users and live streaming rooms. Then, we design two embedding fusion strategies, namely concatenation-based and attention-based methods, to integrate node representations generated by different repeated behaviors. Finally, we adopt a multi-task learning approach to enhance the prediction effectiveness of gift behavior by leveraging predictions from enter behavior and chat behavior. To validate our approach, we construct two live streaming datasets from a large-scale game live streaming platform. Extensive experiments on two real-world datasets with different scales show that our method can significantly outperform various baseline approaches.

Social media popularity prediction is an important channel to explore content sharing and communication on social networks. It aims to capture informative cues by analyzing multi-type data (such as user profile, image, and text) to decide the popularity of a specified post. In this article, we divide social network users into two categories (i.e., active and inactive users) and find a dilemma in existing models: If an active user publishes the low-popularity post, the model will habitually predict the high score. On the contrary, if an inactive user provides the high-popularity post, the model still gives the low score incorrectly. Therefore, how to make the model more subtle to users is important. Comparing to existing methods that directly leverage multi-modal features for regression training, this article stresses more on two novel mechanisms. The first method aims to prevent the over-fitting on user IDs. We propose the attribute-sensitive interactive mechanism (M1) by incorporating explicit user-attribute and post-attribute interaction. It can analyze which type of features a user cares the most and weaken the model’s dependence on user IDs. The second method aims to strengthen the influence of post content. We propose the knowledge embedding mechanism (M2) to revise the popularity scores in existing models by fusing the statistical frequency over multi-type data. Note that both mechanisms are model-agnostic, which can be applicable in any popularity prediction model. Extensive experiments conducted on the Social Media Prediction Dataset further validate the effectiveness.

Expert-designed close-ended benchmarks are indispensable in assessing the knowledge capacity of large language models (LLMs). Despite their widespread use, concerns have mounted regarding their reliability due to limited test scenarios and an unavoidable risk of data contamination. To rectify this, we present PertEval, a toolkit devised for in-depth probing of LLMs' knowledge capacity through knowledge-invariant perturbations. These perturbations employ human-like restatement techniques to generate on-the-fly test samples from static benchmarks, meticulously retaining knowledge-critical content while altering irrelevant details. Our toolkit further includes a suite of response consistency analyses that compare performance on raw vs. perturbed test sets to precisely assess LLMs' genuine knowledge capacity. Six representative LLMs are re-evaluated using PertEval. Results reveal significantly inflated performance of the LLMs on raw benchmarks, including an absolute 25.8% overestimation for GPT-4. Additionally, through a nuanced response pattern analysis, we discover that PertEval retains LLMs' uncertainty to specious knowledge, and reveals their potential rote memorization to correct options which leads to overestimated performance. We also find that the detailed response consistency analyses by PertEval could illuminate various weaknesses in existing LLMs' knowledge mastery and guide the development of refinement. Our findings provide insights for advancing more robust and genuinely knowledgeable LLMs. Our code is available at https://github.com/aigc-apps/PertEval.

Collaborative problem solving (CPS) competence is considered one of the essential 21st-century skills. To facilitate the assessment and learning of CPS competence, researchers have proposed a series of frameworks to conceptualize CPS and explored ways to make sense of the complex processes involved in collaborative problem solving. However, encoding explicit behaviors into subskills within the frameworks of CPS skills is still a challenging task. Traditional studies have relied on manual coding to decipher behavioral data for CPS, but such coding methods can be very time-consuming and cannot support real-time analyses. Scholars have begun to explore approaches for constructing automatic coding models. Nevertheless, the existing models built using machine learning or deep learning techniques depend on a large amount of training data and have relatively low accuracy. To address these problems, this paper proposes a prompt-based learning pre-trained model. The model can achieve high performance even with limited training data. In this study, three experiments were conducted, and the results showed that our model not only produced the highest accuracy, macro F1 score, and kappa values on large training sets, but also performed the best on small training sets of the CPS behavioral data. The application of the proposed prompt-based learning pre-trained model contributes to the CPS skills coding task and can also be used for other CSCW coding tasks to replace manual coding.

Image-text matching, as a fundamental cross-modal task, bridges vision and language. The key challenge lies in accurately learning the semantic similarity of these two heterogeneous modalities. To determine the semantic similarity between visual and textual features, existing paradigm typically first maps them into a d-dimensional shared representation space, then independently aggregates all dimensional correspondences of cross-modal features to reflect it, e.g., the inner product. However, in this paper, we are motivated by an insightful finding that dimensions are not mutually independent, but there are intrinsic dependencies among dimensions to jointly represent latent semantics. Ignoring this intrinsic information probably leads to suboptimal aggregation for semantic similarity, impairing cross-modal matching learning. To solve this issue, we propose a novel cross-dimensional semantic dependency-aware model (called X-Dim), which explicitly and adaptively mines the semantic dependencies between dimensions in the shared space, enabling dimensions with joint dependencies to be enhanced and utilized. X-Dim (1) designs a generalized framework to learn dimensions' semantic dependency degrees, and (2) devises the adaptive sparse probabilistic learning to autonomously make the model capture precise dependencies. Theoretical analysis and extensive experiments demonstrate the superiority of X-Dim over state-of-the-art methods, achieving 5.9%-7.3% rSum improvements on Flickr30K and MS-COCO benchmarks.

Enterprise collaboration technologies (ECTs) are increasingly recognized for supporting effective and efficient digital collaboration, such as decision-making activities, among employees. Given the social and collaborative nature of ECT use, social network theory offers important and helpful insights into how and why employees' social network relations facilitate their ECT use. However, existing research primarily examines the effects of a single social network relation or several social network relations separately, without applying a holistic approach to investigate the joint effect of multiple social network relations on ECT use. Drawing on a novel technique of fuzzy-set qualitative comparative analysis (fsQCA) and social network analysis, this study explores how multiple social network relations (i.e., advice, friendship, and communication) collectively influence ECT use. Using multi-source data from 178 employees in the human resources department of a global technology company, we identify several configurations of multiple social network relations associated with high ECT use and low ECT use. Our findings indicate that a single social network relation is insufficient to explain ECT use and should be considered alongside other social network relations. Overall, this study provides an integrative framework to unpack the complex and contingent effects of multiple social network relations on ECT use.

Cognitive diagnostic assessment is a fundamental task in intelligent education, which aims at quantifying students' cognitive level on knowledge attributes. Since there exists learning dependency among knowledge attributes, it is crucial for cognitive diagnosis models (CDMs) to incorporate attribute hierarchy when assessing students. The attribute hierarchy is only explored by a few CDMs such as Attribute Hierarchy Method, and there are still two significant limitations in these methods. First, the time complexity would be unbearable when the number of attributes is large. Second, the assumption used to model the attribute hierarchy is too strong so that it may lose some information of the hierarchy and is not flexible enough to fit all situations. To address these limitations, we propose a novel Bayesian network-based Hierarchical Cognitive Diagnosis Framework (HierCDF), which enables many traditional diagnostic models to flexibly integrate the attribute hierarchy for better diagnosis. Specifically, we first use an efficient Bayesian network to model the influence of attribute hierarchy on students' cognitive states. Then we design a CDM adaptor to bridge the gap between students' cognitive states and the input features of existing diagnostic models. Finally, we analyze the generality and complexity of HierCDF to show its effectiveness in modeling hierarchy information. The performance of HierCDF is experimentally proved on real-world large-scale datasets.

This paper explores a longitudinal approach to combining engagement, performance and social connectivity data from a MOOC using the framework of exponential random graph models (ERGMs). The idea is to model the social network in the discussion forum in a given week not only using performance (assignment scores) and overall engagement (lecture and discussion views) covariates within that week, but also on the same person-level covariates from adjacent previous and subsequent weeks. We find that over all eight weekly sessions, the social networks constructed from the forum interactions are relatively sparse and lack the tendency for preferential attachment. By analyzing data from the second week, we also find that individuals with higher performance scores from current, previous, and future weeks tend to be more connected in the social network. Engagement with lectures had significant but sometimes puzzling effects on social connectivity. However, the relationships between social connectivity, performance, and engagement weakened over time, and results were not stable across weeks.

Eye tracking is a useful tool to understand students' cognitive process during problem solving. This paper offers a unique perspective by applying techniques from social network analysis to eye movement patterns in mathematics problem solving. We construct and visualize transition networks using eye-tracking data collected from 37 8th grade students while solving linear function problems. By applying network analysis on the constructed transition networks, we find general transition patterns between areas of interest (AOIs) for all students, and we also compare patterns for high- and low-performing students. Our results show that even though students share general transition patterns during problem solving, high-performing students made more strategic transitions among AOI triples than low-performing students.