ChatGPT and other large language models (LLMs) have proven useful in crowdsourcing tasks, where they can effectively annotate machine learning training data. However, this means that they also have the potential for misuse, specifically to automatically answer surveys. LLMs can potentially circumvent quality assurance measures, thereby threatening the integrity of methodologies that rely on crowdsourcing surveys. In this paper, we propose a mechanism to detect LLM-generated responses to surveys. The mechanism uses ''prompt injection,'' such as directions that can mislead LLMs into giving predictable responses. We evaluate our technique against a range of question scenarios, types, and positions, and find that it can reliably detect LLM-generated responses with more than 98% effectiveness. We also provide an open-source software to help survey designers use our technique to detect LLM responses. Our work is a step in ensuring that survey methodologies remain rigorous vis-a-vis LLMs.

Learning to operate a complex system, such as an agile production line, can be a daunting task. The high variability in products and frequent reconfigurations make it difficult to keep documentation up-to-date and share new knowledge amongst factory workers. We introduce CLAICA, a Continuously Learning AI Cognitive Assistant that supports workers in the aforementioned scenario. CLAICA learns from (experienced) workers, formalizes new knowledge, stores it in a knowledge base, along with contextual information, and shares it when relevant. We conducted a user study with 83 participants who performed eight knowledge exchange tasks with CLAICA, completed a survey, and provided qualitative feedback. Our results provide a deeper understanding of how prior training, context expertise, and interaction modality affect the user experience of cognitive assistants. We draw on our results to elicit design and evaluation guidelines for cognitive assistants that support knowledge exchange in fast-paced and demanding environments, such as an agile production line.

Emoji have become an essential part of modern communication, helping to convey emotions and tone quickly and concisely. Emoji used by humans and Intelligent Agents (IA) have been shown to affect people's decision making intentions, suggesting they could be used to manipulate users to follow their advice. We present a mixed-methods crowdsourcing study (N = 194) that shows that adherence to an IA's recommendation and user experience are not affected by emoji when used in a positive, collaborative way. However, we demonstrate that explanations provided by an IA do increase adherence to its recommendation.

The rapid development and ubiquity of mobile and wearable devices promises to enable researchers to monitor users’ granular emotional data in a less intrusive manner. Researchers have used a wide variety of mobile and wearable devices for this purpose, and have proposed various approaches to sense users’ emotional states. In this survey, we utilise three established digital libraries (ACM Digital Library,IEEE Xplore Digital Library, andSpringer Nature). We analysed and critically assessed the different approaches used in the three stages (perception, learning, inference) of a typical mobile emotion sensing framework, following a structured paper selection process. The contribution of this survey is three-fold; first, we document all the latest relevant literature on mobile emotion sensing research; second, we describe how mobile and wearable devices use their sensing and computing capabilities to monitor human emotions; third, we discuss challenges and opportunities of mobile emotion sensing to demonstrate the potential of this thriving field of research.

With the rapid development of mobile and wearable devices, it is increasingly possible to access users’ affective data in a more unobtrusive manner. On this basis, researchers have proposed various systems to recognize user’s emotional states. However, most of these studies rely on traditional machine learning techniques and a limited number of signals, leading to systems that either do not generalize well or would frequently lack sufficient information for emotion detection in realistic scenarios. In this paper, we propose a novel attention-based LSTM system that uses a combination of sensors from a smartphone (front camera, microphone, touch panel) and a wristband (photoplethysmography, electrodermal activity, and infrared thermopile sensor) to accurately determine user’s emotional states. We evaluated the proposed system by conducting a user study with 45 participants. Using collected behavioral (facial expression, speech, keystroke) and physiological (blood volume, electrodermal activity, skin temperature) affective responses induced by visual stimuli, our system was able to achieve an average accuracy of 89.2 percent for binary positive and negative emotion classification under leave-one-participant-out cross-validation. Furthermore, we investigated the effectiveness of different combinations of data signals to cover different scenarios of signal availability.

Non-invasive inspection and imaging techniques are used to acquire non-visible information embedded in samples. Typical applications include medical imaging, defect evaluation, and electronics testing. However, existing methods have specific limitations, including safety risks (e.g., X-ray), equipment costs (e.g., optical tomography), personnel training (e.g., ultrasonography), and material constraints (e.g., terahertz spectroscopy). Such constraints make these approaches impractical for everyday scenarios. In this article, we present a method that is low-cost and practical for non-invasive inspection in everyday settings. Our prototype incorporates a miniaturized near-infrared spectroscopy scanner driven by a computer-controlled 2D-plotter. Our work presents a method to optimize content embedding, as well as a wavelength selection algorithm to extract content without human supervision. We show that our method can successfully extract occluded text through a paper stack of up to 16 pages. In addition, we present a deep-learning-based image enhancement model that can further improve the image quality and simultaneously decompose overlapping content. Finally, we demonstrate how our method can be generalized to different inks and other layered materials beyond paper. Our approach enables a wide range of content embedding applications, including chipless information embedding, physical secret sharing, 3D print evaluations, and steganography.

Recognising and monitoring emotional states play a crucial role in mental health and well-being management. Importantly, with the widespread adoption of smart mobile and wearable devices, it has become easier to collect long-term and granular emotion-related physiological data passively, continuously, and remotely. This creates new opportunities to help individuals manage their emotions and well-being in a less intrusive manner using off-the-shelf low-cost devices. Pervasive emotion recognition based on physiological signals is, however, still challenging due to the difficulty to efficiently extract high-order correlations between physiological signals and users' emotional states. In this paper, we propose a novel end-to-end emotion recognition system based on a convolution-augmented transformer architecture. Specifically, it can recognise users' emotions on the dimensions of arousal and valence by learning both the global and local fine-grained associations and dependencies within and across multimodal physiological data (including blood volume pulse, electrodermal activity, heart rate, and skin temperature). We extensively evaluated the performance of our model using the K-EmoCon dataset, which is acquired in naturalistic conversations using off-the-shelf devices and contains spontaneous emotion data. Our results demonstrate that our approach outperforms the baselines and achieves state-of-the-art or competitive performance. We also demonstrate the effectiveness and generalizability of our system on another affective dataset which used affect inducement and commercial physiological sensors.

Smart speakers have become exceedingly popular and entered many people's homes due to their ability to engage users with natural conversations. Researchers have also looked into using smart speakers as an interface to collect self-reported health data through conversations. Responding to surveys prompted by smart speakers requires users to listen to questions and answer in voice without any visual stimuli. Compared to traditional web-based surveys, where users can see questions and answers visually, voice surveys may be more cognitively challenging. Therefore, to collect reliable survey data, it is important to understand what types of questions are suitable to be administered by smart speakers. We selected five common survey questionnaires and deployed them as voice surveys and web surveys in a within-subject study. Our 24 participants answered questions using voice and web questionnaires in one session. They then repeated the same study session after 1 week to provide a "retest'' response. Our results suggest that voice surveys have comparable reliability to web surveys. We find that, when using 5-point or 7-point scales, voice surveys take about twice as long as web surveys. Based on objective measurements, such as response agreement and test-retest reliability, and subjective evaluations of user experience, we recommend that researchers consider adopting the binary scale and 5-point numerical scales for voice surveys on smart speakers.

We investigate the use of a miniaturized Near-Infrared Spectroscopy (NIRS) device in an assisted decision-making task. We consider the real-world scenario of determining whether food contains gluten, and we investigate how end-users interact with our NIRS detection device to ultimately make this judgment. In particular, we explore the effects of different nutrition labels and representations of confidence on participants’ perception and trust. Our results show that participants tend to be conservative in their judgment and are willing to trust the device in the absence of understandable label information. We further identify strategies to increase user trust in the system. Our work contributes to the growing body of knowledge on how NIRS can be mass-appropriated for everyday sensing tasks, and how to enhance the trustworthiness of assisted decision-making systems.