While multimodal large language models (MLLMs) are increasingly applied in human-centred AI systems, their ability to understand complex social interactions remains uncertain. We present an exploratory study on aligning MLLMs with speech–language pathologists (SLPs) in analysing joint attention in parent–child interactions, a key construct in early social–communicative development. Drawing on interviews and video annotations with three SLPs, we characterise how observational cues of gaze, action, and vocalisation inform their reasoning processes. We then test whether an MLLM can approximate this workflow through a two-stage prompting, separating observation from judgment. Our findings reveal that alignment is more robust at the observation layer, where experts share common descriptors, than at the judgement layer, where interpretive criteria diverge. We position this work as a case-based probe into expert–AI alignment in complex social behaviour, highlighting both the feasibility and the challenges of applying MLLMs to socially situated interaction analysis.

Loss of decisional capacity, coupled with the increasing absence of reliable human proxies, raises urgent questions about how individuals’ values can be represented in Advance Care Planning (ACP). To probe this fraught design space of high-risk, high-subjectivity decision support, we built an experience prototype (ACPAgent) and asked 15 participants in 4 workshops to train it to be their personal ACP proxy. We analysed their coping strategies and feature requests and mapped the results onto axes of agent autonomy and human control. Our findings show a surprising 86.7% agreement with ACPAgent, arguing for a potential new role of AI in ACP where agents act as personal advocates for individuals, building mutual intelligibility over time. We propose that the key areas of future risk that must be addressed are the moderation of users’ expectations and designing accountability and oversight over agent deployment and cutoffs.

Accurate swimming performance monitoring has traditionally relied on wearable sensors, which can disrupt natural technique and are impractical in competitive settings. In this paper, we present a fully vision-based system for automatic swimmer analysis using overhead drone footage, removing the need for any wearable device or underwater equipment. By fine-tuning pose estimation models for aerial aquatic conditions, our approach robustly extracts full-body swimmer skeletons even under challenging scenarios such as splashes and partial occlusions. From these poses, we classify swimming strokes, compute instantaneous speed, estimate lap times, and count individual strokes. Unlike existing methods, our system provides scalable, unobtrusive, and infrastructure-free tracking. Evaluated on real-world drone-captured swimming competition data, our method achieves a median speed estimation error below 4% (under 0.05 m/s), a median lap time error of just 0.03s, and stroke count errors typically under one stroke per lap.

Detecting hate speech and offensive language is essential for maintaining a safe and respectful digital environment.This study examines the limitations of state-of-the-art large language models (LLMs) in identifying offensive content within systematically perturbed data, with a focus on Chinese, a language particularly susceptible to such perturbations.We introduce ToxiCloakCN 1 , an enhanced dataset derived from ToxiCN, augmented with homophonic substitutions and emoji transformations, to test the robustness of LLMs against these cloaking perturbations.Our findings reveal that existing models significantly underperform in detecting offensive content when these perturbations are applied.We provide an in-depth analysis of how different types of offensive content are affected by these perturbations and explore the alignment between human and model explanations of offensiveness.Our work highlights the urgent need for more advanced techniques in offensive language detection to combat the evolving tactics used to evade detection mechanisms.

While AI-assisted individual qualitative analysis has been substantially studied, AI-assisted collaborative qualitative analysis (CQA) – a process that involves multiple researchers working together to interpret data—remains relatively unexplored. After identifying CQA practices and design opportunities through formative interviews, we designed and implemented CoAIcoder, a tool leveraging AI to enhance human-to-human collaboration within CQA through four distinct collaboration methods. With a between-subject design, we evaluated CoAIcoder with 32 pairs of CQA-trained participants across common CQA phases under each collaboration method. Our findings suggest that while using a shared AI model as a mediator among coders could improve CQA efficiency and foster agreement more quickly in the early coding stage, it might affect the final code diversity. We also emphasize the need to consider the independence level when using AI to assist human-to-human collaboration in various CQA scenarios. Lastly, we suggest design implications for future AI-assisted CQA systems.

Recent research has focused on using large language models (LLMs) to generate explanations for hate speech through fine-tuning or prompting. Despite the growing interest in this area, these generated explanations' effectiveness and potential limitations remain poorly understood. A key concern is that these explanations, generated by LLMs, may lead to erroneous judgments about the nature of flagged content by both users and content moderators. For instance, an LLM-generated explanation might inaccurately convince a content moderator that a benign piece of content is hateful. In light of this, we propose an analytical framework for examining hate speech explanations and conducted an extensive survey on evaluating such explanations. Specifically, we prompted GPT-3 to generate explanations for both hateful and non-hateful content, and a survey was conducted with 2,400 unique respondents to evaluate the generated explanations. Our findings reveal that (1) human evaluators rated the GPT-generated explanations as high quality in terms of linguistic fluency, informativeness, persuasiveness, and logical soundness, (2) the persuasive nature of these explanations, however, varied depending on the prompting strategy employed, and (3) this persuasiveness may result in incorrect judgments about the hatefulness of the content. Our study underscores the need for caution in applying LLM-generated explanations for content moderation. Code and results are available at https://github.com/Social-AI-Studio/GPT3-HateEval.

The rapid spread of rumors on social media and their potential impact has motivated the development of automatic rumor detection solutions. However, the existing solutions are mostly limited to detecting rumors in English which neglects the bulk of social media content in other low-resource languages. This paper aims to address the research gaps by proposing Multilingual Source Co-Attention Transformer (MUSCAT), which builds on a multilingual pre-trained language model to perform multilingual rumor detection. Specifically, MUSCAT pivots the source claims in multilingual conversation threads with co-attention transformers to improve detection performance in multilingual settings. We additionally construct multilingual rumor datasets to support our experimental evaluations. Our experimental results show that MUSCAT outperforms state-of-the-art methods in monolingual, cross-lingual, and multilingual rumor detection settings. We have also conducted empirical analysis and outlined the challenges of performing rumor detection in multilingual and cross-lingual settings.

The current COVID-19 pandemic has resulted in disruption to the delivery of higher education. The government-mandated workplace closures that lasted for two months from April 2020 resulted in the closing of all university campuses in our city. This happened while our first-year introductory Python programming course was still in progress. We were thus unable to administer our final exam on campus. In this paper, we describe how our final exam, usually conducted on campus, was replaced with a performance-based assessment. This assessment tasked students to design and program their own game individually. After submitting their code, each student was then required to attend an oral exam that was administered online. We reflect on our experience, drawing from both instructors' and students' perspectives of the programming task and the assessment format. We conclude with a description of how the lessons learnt were applied to a subsequent run of the course.

There exists a multitude of online video tutorials to teach physical movements such as exercises. Yet, users lack support to verify the accuracy of their movements when following such videos and have to rely on their own perception. To address this, we developed a web-based application that performs human pose estimation using both video inputs from the online video and web camera, then provides different types of visual feedback to a user. Our study suggests that a user's skeleton overlaid on the user's camera feed improves user performance, whereas a user's skeleton on its own or trainer's skeleton with the trainer video offered limited benefits. Our application demonstrates the potential to enhance learning physical movements from online videos and provides a basis for other guidance systems to design suitable visualizations.

With mobile apps rapidly permeating all aspects of daily living with use by all segments of the population, it is crucial to support the evaluation of app usability for specific impaired users to improve app accessibility. In this work, we examine the effects of using our augmented virtuality impairment simulation system--Empath-D--to support experienced designer-developers to redesign a mockup of commonly used mobile application for cataract-impaired users, comparing this with existing tools that aid designing for accessibility. We show that the use of augmented virtuality for assessing usability supports enhanced usability challenge identification, finding more defects and doing so more accurately than with existing methods. Through our user interviews, we also show that augmented virtuality impairment simulation supports realistic interaction and evaluation to provide a concrete understanding over the usability challenges that impaired users face, and complements the existing guidelines-based approaches meant for general accessibility.

With app-based interaction increasingly permeating all aspects of daily living, it is essential to ensure that apps are designed to be inclusive and are usable by a wider audience such as the elderly, with various impairments (e.g., visual, audio and motor). We propose Empath-D, a system that fosters empathetic design, by allowing app designers, in-situ, to rapidly evaluate the usability of their apps, from the perspective of impaired users. To provide a truly authentic experience, Empath-D carefully orchestrates the interaction between a smartphone and a VR device, allowing the user to experience simulated impairments in a virtual world while interacting naturally with the app, using a real smartphone. By carefully orchestrating the VR-smartphone interaction, Empath-D tackles challenges such as preserving low-latency app interaction, accurate visualization of hand movement and low-overhead perturbation of I/O streams. Experimental results show that user interaction with Empath-D is comparable (both in accuracy and user perception) to real-world app usage, and that it can simulate impairment effects as effectively as a custom hardware simulator.

No abstract available.