AI summaries on social media are reshaping how users form opinions about political topics, yet their influence remains largely unexamined despite their widespread deployment. This paper investigates how two types of AI summaries affect user opinions and engagement: textual summaries of discussion narratives and percentage breakdowns of agreement/disagreement. Through a 144-participant experiment on simulated online discussion threads, we found that displaying commenter agreement percentages amplified social conformity towards the majority views beyond reading comments alone. Conversely, AI narrative summaries created misperceptions of balance in polarised threads, reducing opinion change. While these summaries did not influence participants’ willingness to engage, toxic discussions deterred participation even when participants held majority views. Based on our findings, we provide critical design interventions for industry and researchers to mitigate these tools’ polarising effects, paving the way for responsible AI deployment on social media platforms.

Short-form video platforms have become major channels for misinformation, with their rich multimodal features making false claims highly believable. HCI research shows that providing corrections in the same modality as the misinformation can be an effective solution. However, since corrections and misinformation convey contradicting information, the order in which one is exposed to them can impact what one believes. We conducted a between-subjects mixed-methods experiment where participants (N=120) rated the credibility of misinformation statements before and after viewing misinformation videos paired with correction videos. Corrections were shown either before, during, or after misinformation. Across all three timings, corrections reduced belief in misinformation, but post-exposure corrections proved most effective and mid-exposure corrections least effective. These findings suggest that correction mechanisms should appear after misinformation exposure, while avoiding mid-exposure interruptions that reduce impact. We outline design recommendations for integrating correction videos into short-form video platforms to improve resilience against misinformation.

In human–robot collaboration, repeated failures are inevitable and can undermine trust and perceptions of robot intelligence. While some failures severely disrupt tasks and others are relatively benign, their cumulative impact on trust is not clearly understood. We investigated whether users perceive repeated failures of the same type differently from varied failures, and how robot awareness of its own failures affects these perceptions. In a collaborative physical task with 54 participants, we manipulated failure sequence (homogeneous vs. heterogeneous) and awareness (none, partial, full). Results show that trust and perceived intelligence were influenced by both current and prior failures, with homogeneous sequences leading to smaller reductions in these evaluations compared to heterogeneous ones. Robots displaying awareness, whether partial or full, were consistently rated higher than unaware robots, particularly for grasping and planning failures. Our findings provide a deeper understanding of how failure type, sequence, and robot awareness shape users’ perceptions of collaborative robots.

Mental health challenges are increasingly prevalent among university students, yet often go undetected due to reliance on traditional assessments that are subjective, infrequent, and lack behavioral context. Digital phenotyping through passively collected smartphone data offers a scalable alternative, but existing approaches often fail to integrate predictive accuracy with narrative-based insights. To overcome these limitations, we present NarrativeSense, a novel framework that combines machine learning models with narrative-based descriptions of daily life events inferred from smartphone sensing data to predict weekly affective states. The system incorporates language model components to transform behavioral patterns into contextualized, human-readable narratives that ground affective predictions in everyday experiences. This narrative layer complements structured prediction by offering intuitive, user-centered insights. Applied to longitudinal data from 58 university students over 119 days, NarrativeSense outperforms baseline machine learning models, standalone LLMs, and ensemble methods, while providing richer insights. Our findings demonstrate the potential of narrative-enhanced digital phenotyping for scalable and explainable mental health monitoring in educational and clinical settings.

Smartphones are essential to daily life, and their rich data streams have been used to study how people use their phones, and more broadly human behaviour. While previous research has largely focused on app usage and keystroke dynamics to predict smartphone use, these analyses are typically limited to making predictions rather than providing explanations or reasoning for observed behaviours. In this exploratory study, we investigate the potential of leveraging screen text and large language models (LLMs) to uncover insights and reasoning about user behaviour. Using a dataset of over 100 million on-screen words collected from 21 participants over two weeks, we explore multiple ways to use screen text and LLMs for three tasks: predicting the next app a user will open, inferring what real-world activities they are engaged in, and understanding how they interact within apps. Orthogonally, we demonstrate the interpretive capabilities of LLMs, highlighting their potential to explain the reasoning behind observed user actions. Our findings suggest that screen text holds promise for providing deeper insights into both digital and real-world human behaviour. We discuss the broader implications of our findings, including enhancing user experience and enabling privacy-preserving, on-device analysis, while proposing future research directions in screen text analysis.

While short video platforms such as TikTok, YouTube Shorts, and Instagram Reels are frequently criticised for facilitating the spread of misinformation, they are also increasingly leveraged as tools for countering it through debunking content. Although video-based corrections have demonstrated effectiveness, their persuasive impact may depend on the richness of their audio-visual elements. This study examines the persuasive efficacy of three fundamental presentation styles commonly used in short-form video content: (1) videos featuring only captions, (2) captions accompanied by relevant images, and (3) captions presented alongside the creator’s visible face. Our results indicate that videos incorporating either relevant and engaging imagery or the creator’s facial presence are significantly more persuasive than those relying solely on captions. Based on these findings, we propose practical recommendations for improving the effectiveness of debunking videos, with the aim of promoting belief revision and mitigating misinformation on short video platforms.

Robots are prone to making errors, which can negatively impact their credibility as teammates during collaborative tasks with human users. Detecting and recovering from these failures is crucial for maintaining effective level of trust from users. However, robots may fail without being aware of it. One way to detect such failures could be by analysing humans' non-verbal behaviours and reactions to failures. This study investigates how human gaze dynamics can signal a robot's failure and examines how different types of failures affect people's perception of robot. We conducted a user study with 27 participants collaborating with a robotic mobile manipulator to solve tangram puzzles. The robot was programmed to experience two types of failures -executional and decisional- occurring either at the beginning or end of the task, with or without acknowledgement of the failure. Our findings reveal that the type and timing of the robot's failure significantly affect participants' gaze behaviour and perception of the robot. Specifically, executional failures led to more gaze shifts and increased focus on the robot, while decisional failures resulted in lower entropy in gaze transitions among areas of interest, particularly when the failure occurred at the end of the task. These results highlight that gaze can serve as a reliable indicator of robot failures and their types, and could also be used to predict the appropriate recovery actions.

Detecting robot failures during collaborative tasks is crucial for maintaining trust in human-robot interactions. This study investigates user gaze behaviour as an indicator of robot failures, utilising machine learning models to distinguish between non-failure and two types of failures: executional and decisional. Eye-tracking data were collected from 26 participants collaborating with a robot on Tangram puzzle-solving tasks. Gaze metrics, such as average gaze shift rates and the probability of gazing at specific areas of interest, were used to train machine learning classifiers, including Random Forest, AdaBoost, XGBoost, SVM, and CatBoost. The results show that Random Forest achieved 90 % accuracy for detecting executional failures and 80 % for decisional failures using the first 5 seconds of failure data. Real-time failure detection was evaluated by segmenting gaze data into intervals of 3, 5, and 10 seconds. These findings highlight the potential of gaze dynamics for real-time error detection in human-robot collaboration.

ChatGPT and other large language models (LLMs) have proven useful in crowdsourcing tasks, where they can effectively annotate machine learning training data. However, this means that they also have the potential for misuse, specifically to automatically answer surveys. LLMs can potentially circumvent quality assurance measures, thereby threatening the integrity of methodologies that rely on crowdsourcing surveys. In this paper, we propose a mechanism to detect LLM-generated responses to surveys. The mechanism uses ''prompt injection,'' such as directions that can mislead LLMs into giving predictable responses. We evaluate our technique against a range of question scenarios, types, and positions, and find that it can reliably detect LLM-generated responses with more than 98% effectiveness. We also provide an open-source software to help survey designers use our technique to detect LLM responses. Our work is a step in ensuring that survey methodologies remain rigorous vis-a-vis LLMs.

Implantable Medical Devices (IMDs) are widely deployed today and often use wireless communication. Establishing a secure communication channel to these devices is challenging in practice. To address this issue, researchers have proposed IMD key exchange protocols, particularly ones that leverage an Out-Of-Band (OOB) channel such as audio, vibration and physiological signals. While these solutions have advantages over traditional key exchange, they are often proposed in an ad-hoc manner and lack a systematic evaluation of their security, usability and deployability properties. In this paper, we provide an in-depth analysis of existing OOB-based solutions for IMDs and, based on our findings, propose a novel IMD key exchange protocol that includes a new class of OOB channel based on human bodily motions. We implement prototypes and validate our designs through a user study (N = 24). The results demonstrate the feasibility of our approach and its unique features, establishing a new direction in the context of IMD security.

Online polarisation can tear the fabric of civility through reinforcing social media’s perceptions of division and discord. Social media platforms often rely on content-moderation to combat polarisation, contingent on the reactive removal or flagging of content. However, this approach often remains agnostic of the underlying debate’s ideas and stifles open discourse. In this study, we use prompt-tuned language models to mediate social media debates, applying the strategies of the Thomas-Kilmann Conflict Mode Instrument (TKI). We evaluate multiple mediation strategies in providing targeted responses to the debates, as shown to a debate audience. Our findings show that high-cooperativeness TKI strategies offered more persuasive arguments, while an accommodating argument strategy was the most successful at depolarising the audience’s opinion. Furthermore, high-cooperativeness strategies also increased the perception that the debaters will reach a consensus. Our work paves the way for scalable and personalised tools that mediate social media debates to encourage depolarisation.

Context sensing on smartphones is often used to understand user behaviour. Amongst the many available sensors, the collection of text is crucial due to its richness. However, previous work has been limited to collecting text only from keyboard input, or intermittently collecting screen text indirectly by taking screenshots and applying optical character recognition. Here, we present a novel software sensor that unobtrusively and continuously captures all screen text on smartphones. We conducted a validation study with 21 participants over a two-week period, where they used our software on their personal smartphones. Our findings demonstrate how data from our sensor can be used to understand user behaviour and categorise mobile apps. We also show how smartphone sensing can be enhanced by using our sensor in conjunction with other sensors. We discuss the strengths and limitations of our sensor, highlighting potential areas for improvement and providing recommendations for its use.

Healthcare monitoring is crucial for early detection, timely intervention, and the ongoing management of health conditions, ultimately improving individuals' quality of life. Recent research shows that Large Language Models (LLMs) have demonstrated impressive performance in supporting healthcare tasks. However, existing LLM-based healthcare solutions typically rely on cloud-based systems, which raise privacy concerns and increase the risk of personal information leakage. As a result, there is growing interest in running these models locally on devices like mobile phones and wearables to protect users' privacy. Small Language Models (SLMs) are potential candidates to solve privacy and computational issues, as they are more efficient and better suited for local deployment. However, the performance of SLMs in healthcare domains has not yet been investigated. This paper examines the capability of SLMs to accurately analyze health data, such as steps, calories, sleep minutes, and other vital statistics, to assess an individual's health status. Our results show that, TinyLlama, which has 1.1 billion parameters, utilizes 4.31 GB memory, and has 0.48s latency, showing the best performance compared other four state-of-the-art (SOTA) SLMs on various healthcare applications. Our results indicate that SLMs could potentially be deployed on wearable or mobile devices for real-time health monitoring, providing a practical solution for efficient and privacy-preserving healthcare.

Journaling offers significant benefits, including fostering self-reflection, enhancing writing skills, and aiding in mood monitoring. However, many people abandon the practice because traditional journaling is time-consuming, and detailed life events may be overlooked if not recorded promptly. Given that smartphones are the most widely used devices for entertainment, work, and socialization, they present an ideal platform for innovative approaches to journaling. Despite their ubiquity, the potential of using digital phenotyping, a method of unobtrusively collecting data from digital devices to gain insights into psychological and behavioral patterns, for automated journal generation has been largely underexplored. In this study, we propose AutoJournaling, the first-of-its-kind system that automatically generates journals by collecting and analyzing screenshots from smartphones. This system captures life events and corresponding emotions, offering a novel approach to digital phenotyping. We evaluated AutoJournaling by collecting screenshots every 3 seconds from three students over five days, demonstrating its feasibility and accuracy. AutoJournaling is the first framework to utilize seamlessly collected screenshots for journal generation, providing new insights into psychological states through digital phenotyping.

MicroService Architecture (MSA) is gaining rapid popularity for developing large-scale IoT applications for deployment within distributed and resource-constrained Fog computing environments. As a cloud-native application architecture, the true power of microservices comes from their loosely coupled, independently deployable and scalable nature, enabling distributed placement and dynamic composition across federated Fog and Cloud clusters. Thus, it is necessary to develop novel placement algorithms that utilise these microservice characteristics to improve the performance of the applications. However, existing Fog computing frameworks lack support for integrating such placement policies due to their shortcomings in multiple areas, including MSA application placement and deployment across multi-fog multi-cloud environments, dynamic microservice composition across multiple distributed clusters, scalability of the framework to operate within federated environments, support for deploying heterogeneous microservice applications, etc. To this end, we design and implement MicroFog, a Fog computing framework compatible with cloud-native technologies such as Docker, Kubernetes and Istio. MicroFog provides an extensible and configurable control engine that executes placement algorithms and deploys applications across federated Fog environments. Furthermore, MicroFog provides a sufficient abstraction over container orchestration and dynamic microservice composition, thus enabling users to easily incorporate new placement policies and evaluate their performance. The capabilities of the MicroFog framework, such as the scalability and flexibility of the design and deployment architecture of MicroFog and its ability to ensure the deployment and composition of microservices across distributed fog-cloud environments, are validated using multiple use cases. Experiments also demonstrate MicroFog’s ability to integrate and evaluate novel placement policies and load-balancing techniques. To this end, we integrate multiple microservice placement policies to demonstrate MicroFog’s ability to support horizontally scaled placement, service discovery and load balancing of microservices across federated environments, thus reducing the application service response time up to 54%.

While UbiComp research has steadily improved the performance of localisation systems, the analysis of such datasets remains largely unaddressed. In this paper, we present a tool to facilitate querying and analysis of localisation time-series with a focus on semantic localisation. Drawing on well-established models to represent movement and mobility, we first develop a query language for localisation datasets. We then develop a software library in R that implements this querying. We use case studies to demonstrate how our programming tool can be used to query localisation datasets. Our work addresses an important gap in localisation research, by providing a flexible tool that can model and analyse localisation data programmatically and in real time.

The fog computing paradigm is rapidly gaining popularity for latency-critical and bandwidth-hungry IoT application deployment. Meanwhile, MicroService Architecture (MSA) is increasingly adopted for developing IoT applications due to its high scalability and extensibility. For mission-critical IoT services in fog, reliability remains one of the most critical QoS requirements due to less dependability of fog resources. Granular microservices with independent deployment and scaling exhibit great potential in utilising resource-constrained fog resources to improve reliability through redundant placement. However, current research on service placement lacks reliability-aware holistic approaches that combine the MSA features and failure characteristics of fog resources under independent and correlated failures. Hence, we analyse MSA and formulate the reliability-aware placement problem by modelling composite services as k-out-of-n serial-parallel systems in a throughput-aware manner for placement under fog resource failures. Our proposed Reliability-aware Placement Method (RPM) is a hierarchical policy combining improved PSO and NSGA-II algorithms. We integrate it with Monte Carlo reliability calculations to produce redundant placements reaching a trade-off between reliability and cost. The performance results reveal that compared to the benchmarks, our algorithm shows significant improvements in reliability satisfaction (up to 25%) and time to first failure (up to 40%), thus providing a robust placement method.

Non-invasive inspection and imaging techniques are used to acquire non-visible information embedded in samples. Typical applications include medical imaging, defect evaluation, and electronics testing. However, existing methods have specific limitations, including safety risks (e.g., X-ray), equipment costs (e.g., optical tomography), personnel training (e.g., ultrasonography), and material constraints (e.g., terahertz spectroscopy). Such constraints make these approaches impractical for everyday scenarios. In this article, we present a method that is low-cost and practical for non-invasive inspection in everyday settings. Our prototype incorporates a miniaturized near-infrared spectroscopy scanner driven by a computer-controlled 2D-plotter. Our work presents a method to optimize content embedding, as well as a wavelength selection algorithm to extract content without human supervision. We show that our method can successfully extract occluded text through a paper stack of up to 16 pages. In addition, we present a deep-learning-based image enhancement model that can further improve the image quality and simultaneously decompose overlapping content. Finally, we demonstrate how our method can be generalized to different inks and other layered materials beyond paper. Our approach enables a wide range of content embedding applications, including chipless information embedding, physical secret sharing, 3D print evaluations, and steganography.

Implicit tendencies and cognitive biases play an important role in how information is perceived and processed, a fact that can be both utilised and exploited by computing systems. The Implicit Association Test (IAT) has been widely used to assess people’s associations of target concepts with qualitative attributes, such as the likelihood of being hired or convicted depending on race, gender, or age. The condensed version–the Brief IAT–aims to implicit biases by measuring the reaction time to concept classifications. To use this measure in HCI research, however, we need a way to construct and validate target concepts, which tend to quickly evolve and depend on geographical and cultural interpretations. In this paper, we introduce and evaluate a new method to appropriate the BIAT using crowdsourcing to measure people’s leanings on polarising topics. We present a web-based tool to test participants’ bias on custom themes, where self-assessments often fail. We validated our approach with 14 domain experts and assessed the fit of crowdsourced test construction. Our method allows researchers of different domains to create and validate bias tests that can be geographically tailored and updated over time. We discuss how our method can be applied to surface implicit user biases and run studies where cognitive biases may impede reliable results.

Two decades of focus on User Experience has yielded an array of digital technologies that help people experience, understand and share emotions. Although the effects of specific technologies upon emotion have been well studied, less is known about how people actively appropriate and combine the full range of devices, apps and services at their disposal to deliberately manage emotions in everyday life. We conducted a one-week diary study in which 23 adults recorded interactions between their emotions and technology use. They reported using a diverse range of emotion-shaping tools and strategies as part of coping with daily challenges, managing routines, and pursuing work and social goals. We analyse these data in the light of psychological theories of emotion. Our findings point to the significance of digital emotion regulation as a powerful perspective to inform wider debates about the impacts of technology on social and emotional well-being.

Voice user interfaces (VUIs) have made their way into people’s daily lives, from voice assistants to smart speakers. Although VUIs typically just react to direct user commands, increasingly, they incorporate elements of proactive behaviors. In particular, proactive smart speakers have the potential for many applications, ranging from healthcare to entertainment; however, their usability in everyday life is subject to interaction errors. To systematically investigate the nature of errors, we designed a voice-based Experience Sampling Method (ESM) application to run on proactive speakers. We captured 1,213 user interactions in a 3-week field deployment in 13 participants’ homes. Through auxiliary audio recordings and logs, we identify substantial interaction errors and strategies that users apply to overcome those errors. We further analyze the interaction timings and provide insights into the time cost of errors. We find that, even for answering simple ESMs, interaction errors occur frequently and can hamper the usability of proactive speakers and user experience. Our work also identifies multiple facets of VUIs that can be improved in terms of the timing of speech.

Smart speakers have become exceedingly popular and entered many people's homes due to their ability to engage users with natural conversations. Researchers have also looked into using smart speakers as an interface to collect self-reported health data through conversations. Responding to surveys prompted by smart speakers requires users to listen to questions and answer in voice without any visual stimuli. Compared to traditional web-based surveys, where users can see questions and answers visually, voice surveys may be more cognitively challenging. Therefore, to collect reliable survey data, it is important to understand what types of questions are suitable to be administered by smart speakers. We selected five common survey questionnaires and deployed them as voice surveys and web surveys in a within-subject study. Our 24 participants answered questions using voice and web questionnaires in one session. They then repeated the same study session after 1 week to provide a "retest'' response. Our results suggest that voice surveys have comparable reliability to web surveys. We find that, when using 5-point or 7-point scales, voice surveys take about twice as long as web surveys. Based on objective measurements, such as response agreement and test-retest reliability, and subjective evaluations of user experience, we recommend that researchers consider adopting the binary scale and 5-point numerical scales for voice surveys on smart speakers.

We investigate the use of a miniaturized Near-Infrared Spectroscopy (NIRS) device in an assisted decision-making task. We consider the real-world scenario of determining whether food contains gluten, and we investigate how end-users interact with our NIRS detection device to ultimately make this judgment. In particular, we explore the effects of different nutrition labels and representations of confidence on participants’ perception and trust. Our results show that participants tend to be conservative in their judgment and are willing to trust the device in the absence of understandable label information. We further identify strategies to increase user trust in the system. Our work contributes to the growing body of knowledge on how NIRS can be mass-appropriated for everyday sensing tasks, and how to enhance the trustworthiness of assisted decision-making systems.

Researchers in Human-Computer Interaction typically rely on experiments to assess the causal effects of experimental conditions on variables of interest. Although this classic approach can be very useful, it offers little help in tackling questions of causality in the kind of data that are increasingly common in HCI – capturing user behavior ‘in the wild.’ To analyze such data, model-based regressions such as cross-lagged panel models or vector autoregressions can be used, but these require parametric assumptions about the structural form of effects among the variables. To overcome some of the limitations associated with experiments and model-based regressions, we adopt and extend ‘empirical dynamic modelling’ methods from ecology that lend themselves to conceptualizing multiple users’ behavior as complex nonlinear dynamical systems. Extending a method known as ‘convergent cross mapping’ or CCM, we show how to make causal inferences that do not rely on experimental manipulations or model-based regressions and, by virtue of being non-parametric, can accommodate data emanating from complex nonlinear dynamical systems. By using this approach for multiple users, which we call ‘multiple convergent cross mapping’ or MCCM, researchers can achieve a better understanding of the interactions between users and technology – by distinguishing causality from correlation – in real-world settings.

In the context of learning systems, identifying causal relationships among information presented to the user, their behavior and cognitive effort required/exerted to understand and perform a task is key to building effective learning experiences, and to maintain engagement in learning processes. An unexplored question is whether our interaction with presented information affects our cognitive effort (and behaviour), or vice-versa. We investigate causal relationship between information presented and cognitive effort (and behaviour) in the context of two separate studies (N = 40, N = 98), and study the effect of instruction (active/passive task). We utilize screen-recordings and eye-tracking data to investigate the relationship among these variables. To investigate the causal relationships among the different measurements, we use Granger’s causality. Further, we propose a new method to combine two time-series from multiple participants for detecting causal relationships. Our results indicate that information presentation drives user focus size (behaviour), and that cognitive load (a measure of cognitive effort exerted) drives information presentation. This relationship is also moderated by instruction type and performance-level (high/low). We draw implications for design of educational material and learning technologies.

Abstract Despite large investments in smartwatch development, the market growth remains smaller than forecasted. The purpose of smartwatch use remains unclear, indicated by the lack of large-scale adoption. Thus, we aim to better understand the early adoption and everyday smartwatch use. We investigate a diverse usage data of smartwatches logged over a period of up to 14 months from 79 individuals between December 2015 and March 2017, one of the largest wearable datasets collected. First, we identify both explorative and accepted behaviours that users exhibit and further investigate how the individual usage traits and features differ between the two categories. Our analysis offers an insightful perspective on how smartwatch use evolves organically. Our results improve our shared understanding of smartwatch use and users adapting their use of smartwatch over time to match the capabilities of the technology by validating numerous findings from previous literature.

Understanding human mobility benefits numerous applications such as urban planning, traffic control, and city management. Previous work mainly focuses on modeling spatial and temporal patterns of human mobility. However, the semantics of trajectory are ignored, thus failing to model people's motivation behind mobility. In this paper, we propose a novel semantics-aware mobility model that captures human mobility motivation using large-scale semantic-rich spatial-temporal data from location-based social networks. In our system, we first develop a multimodal embedding method to project user, location, time, and activity on the same embedding space in an unsupervised way while preserving original trajectory semantics. Then, we use hidden Markov model to learn latent states and transitions between them in the embedding space, which is the location embedding vector, to jointly consider spatial, temporal, and user motivations. In order to tackle the sparsity of individual mobility data, we further propose a von Mises-Fisher mixture clustering for user grouping so as to learn a reliable and fine-grained model for groups of users sharing mobility similarity. We evaluate our proposed method on two large-scale real-world datasets, where we validate the ability of our method to produce high-quality mobility models. We also conduct extensive experiments on the specific task of location prediction. The results show that our model outperforms state-of-the-art mobility models with higher prediction accuracy and much higher efficiency.

Inspired by the increasing prevalence of digital voice assistants, we demonstrate the feasibility of using voice interfaces to deploy and complete crowd tasks. We have developed Crowd Tasker, a novel system that delivers crowd tasks through a digital voice assistant. In a lab study, we validate our proof-of-concept and show that crowd task performance through a voice assistant is comparable to that of a web interface for voice-compatible and voice-based crowd tasks for native English speakers. We also report on a field study where participants used our system in their homes. We find that crowdsourcing through voice can provide greater flexibility to crowd workers by allowing them to work in brief sessions, enabling multi-tasking, and reducing the time and effort required to initiate tasks. We conclude by proposing a set of design guidelines for the creation of crowd tasks for voice and the development of future voice-based crowdsourcing systems.

In this paper, we demonstrate the existence of a bidirectional causal relationship between smartphone application use and user emotions. In a two-week long in-the-wild study with 30 participants we captured 502,851 instances of smartphone application use in tandem with corresponding emotional data from facial expressions. Our analysis shows that while in most cases application use drives user emotions, multiple application categories exist for which the causal effect is in the opposite direction. Our findings shed light on the relationship between smartphone use and emotional states. We furthermore discuss the opportunities for research and practice that arise from our findings and their potential to support emotional well-being.

Although crowd work is typically completed through desktop or laptop computers by workers at their home, literature has shown that crowdsourcing is feasible through a wide array of computing devices, including smartphones and digital voice assistants. An integrated crowdsourcing platform that operates across multiple devices could provide greater flexibility to workers, but there is little understanding of crowd workers’ perceptions on uptaking crowd tasks across multiple contexts through such devices. Using a crowdsourcing survey task, we investigate workers’ willingness to accept different types of crowd tasks presented on three device types in different scenarios of varying location, time and social context. Through analysis of over 25,000 responses received from 329 crowd workers on Amazon Mechanical Turk, we show that when tasks are presented on different devices, the task acceptance rate is 80.5% on personal computers, 77.3% on smartphones and 70.7% on digital voice assistants. Our results also show how different contextual factors such as location, social context and time influence workers decision to accept a task on a given device. Our findings provide important insights towards the development of effective task assignment mechanisms for cross-device crowd platforms.

The assessment of learning during class activities mostly relies on standardized questionnaires to evaluate the efficacy of the learning design elements. However, standardized questionnaires pose additional strain on students, do not provide “temporal” information during the learning experience, require considerable effort and language competence, and sometimes are not appropriate. To overcome these challenges, we propose using wearable devices, which allow for continuous and unobtrusive monitoring of physiological parameters during learning. In this paper we set out to quantify how well we can infer students’ learning experience from wrist-worn devices capturing physiological data. We collected data from 31 students in 93 class sessions (3 class sessions per student), and our analysis shows that wrist data can predict the learning experience with 11% error. We also show that 6.25 min (SD = 3.1 min) of data are needed to achieve a reliable estimate (i.e., 13.8% error). Our work highlights the benefits and limitations of utilizing wearable devices to assess learning experiences. Our findings help shape the future of quantified-self technologies in learning by pointing out the substantial benefits of physiological sensing for self-monitoring, evaluation, and metacognitive reflection in learning.

While crowd workers typically complete a variety of tasks in crowdsourcing platforms, there is no widely accepted method to successfully match workers to different types of tasks. Researchers have considered using worker demographics, behavioural traces, and prior task completion records to optimise task assignment. However, optimum task assignment remains a challenging research problem due to limitations of proposed approaches, which in turn can have a significant impact on the future of crowdsourcing. We present 'CrowdCog', an online dynamic system that performs both task assignment and task recommendations, by relying on fast-paced online cognitive tests to estimate worker performance across a variety of tasks. Our work extends prior work that highlights the effect of workers' cognitive ability on crowdsourcing task performance. Our study, deployed on Amazon Mechanical Turk, involved 574 workers and 983 HITs that span across four typical crowd tasks (Classification, Counting, Transcription, and Sentiment Analysis). Our results show that both our assignment method and recommendation method result in a significant performance increase (5% to 20%) as compared to a generic or random task assignment. Our findings pave the way for the use of quick cognitive tests to provide robust recommendations and assignments to crowd workers.

Social conformity occurs when individuals in group settings change their personal opinion to be in agreement with the majority's position. While recent literature frequently reports on conformity in online group settings, the causes for online conformity are yet to be fully understood. This study aims to understand how social presencei.e., the sense of being connected to others via mediated communication, influences conformity among individuals placed in online groups while answering subjective and objective questions. Acknowledging its multifaceted nature, we investigate three aspects of online social presence: user representation (generic vs.user-specific avatars), interactivity (discussion vs.no discussion ), and response visibility (public vs.private ). Our results show an overall conformity rate of 30% and main effects from task objectivity, group size difference between the majority and the minority, and self-confidence on personal answer. Furthermore, we observe an interaction effect between interactivity and response visibility, such that conformity is highest in the presence of peer discussion and public responses, and lowest when these two elements are absent. We conclude with a discussion on the implications of our findings in designing online group settings, accounting for the effects of social presence on conformity.

Mobile self-reports are a popular technique to collect participant labelled data in the wild. While literature has focused on increasing participant compliance to self-report questionnaires, relatively little work has assessed response accuracy. In this paper, we investigate how participant context can affect response accuracy and help identify strategies to improve the accuracy of mobile self-report data. In a 3-week study we collect over 2,500 questionnaires containing both verifiable and non-verifiable questions. We find that response accuracy is higher for questionnaires that arrive when the phone is not in ongoing or very recent use. Furthermore, our results show that long completion times are an indicator of a lower accuracy. Using contextual mechanisms readily available on smartphones, we are able to explain up to 13% of the variance in participant accuracy. We offer actionable recommendations to assist researchers in their future deployments of mobile self-report studies.

A fundamental challenge in real-time labelling of activity data is user burden. The Experience Sampling Method (ESM) is widely used to obtain such labels for sensor data. However, in an in-situ deployment, it is not feasible to expect users to precisely label the start and end time of each event or activity. For this reason, time-point based experience sampling (without an actual start and end time) is prevalent. We present a framework that applies multi-instance and semi-supervised learning techniques to perform to predict user annotations from multiple mobile sensor data streams. Our proposed framework estimates users' annotations in ESM-based studies progressively, via an interactive pipeline of co-training and active learning. We evaluate our work using data collected from an in-the-wild data collection.

Understanding human mobility benefits numerous applications such as urban planning, traffic control and city management. Previous work mainly focuses on modeling spatial and temporal patterns of human mobility. However, the semantics of trajectory are ignored, thus failing to model people's motivation behind mobility. In this paper, we propose a novel semantics-aware mobility model that captures human mobility motivation using large-scale semantics-rich spatial-temporal data from location-based social networks. In our system, we first develop a multimodal embedding method to project user, location, time, and activity on the same embedding space in an unsupervised way while preserving original trajectory semantics. Then, we use hidden Markov model to learn latent states and transitions between them in the embedding space, which is the location embedding vector, to jointly consider spatial, temporal, and user motivations. In order to tackle the sparsity of individual mobility data, we further propose a von Mises-Fisher mixture clustering for user grouping so as to learn a reliable and fine-grained model for groups of users sharing mobility similarity. We evaluate our proposed method on two large-scale real-world datasets, where we validate the ability of our method to produce high-quality mobility models. We also conduct extensive experiments on the specific task of location prediction. The results show that our model outperforms state-of-the-art mobility models with higher prediction accuracy and much higher efficiency.

The increased reliance on algorithmic decision-making in socially impactful processes has intensified the calls for algorithms that are unbiased and procedurally fair. Identifying fair predictors is an essential step in the construction of equitable algorithms, but the lack of ground-truth in fair predictor selection makes this a challenging task. In our study, we recruit 90 crowdworkers to judge the inclusion of various predictors for recidivism. We divide participants across three conditions with varying group composition. Our results show that participants were able to make informed decisions on predictor selection. We find that agreement with the majority vote is higher when participants are part of a more diverse group. The presented workflow, which provides a scalable and practical approach to reach a diverse audience, allows researchers to capture participants' perceptions of fairness in private while simultaneously allowing for structured participant discussion.

Social conformity occurs when an individual changes their behaviour in line with the majority's expectations. Although social conformity has been investigated in small group settings, the effect of gender - of both the individual and the majority/minority - is not well understood in online settings. Here we systematically investigate the impact of groups' gender composition on social conformity in online settings. We use an online quiz in which participants submit their answers and confidence scores, both prior to and following the presentation of peer answers that are dynamically fabricated. Our results show an overall conformity rate of 39%, and a significant effect of gender that manifests in a number of ways: gender composition of the majority, the perceived nature of the question, participant gender, visual cues of the system, and final answer correctness. We conclude with a discussion on the implications of our findings in designing online group settings, accounting for the effects of gender on conformity.

Low back pain (LBP) is a globally common condition with no silver bullet solutions. Further, the lack of therapeutic consensus causes challenges in choosing suitable solutions to try. In this work, we crowdsourced knowledge bases on LBP treatments. The knowledge bases were used to rank and offer best-matching LBP treatments to end users. We collected two knowledge bases: one from clinical professionals and one from non-professionals. Our quantitative analysis revealed that non-professional end users perceived the best treatments by both groups as equally good. However, the worst treatments by non-professionals were clearly seen as inferior to the lowest ranking treatments by professionals. Certain treatments by professionals were also perceived significantly differently by non-professionals and professionals themselves. Professionals found our system handy for self-reflection and for educating new patients, while non-professionals appreciated the reliable decision support that also respected the non-professional opinion.

IoT services hosted by low-power devices rely on the cloud infrastructure to propagate their ubiquitous presence over the Internet. A critical challenge for IoT systems is to ensure continuous provisioning of IoT services by overcoming network breakdowns, hardware failures, and energy constraints. To overcome these issues, we propose a cloud-based framework namely SensorClone, which relies on virtual devices to improve IoT resilience. A virtual device is the digital counterpart of a physical device that has learned to emulate its operations from sample data collected from the physical one. SensorClone exploits the collected data of low-power devices to create virtual devices in the cloud. SensorClone then can opportunistically migrate virtual devices from the cloud into other devices, potentially underutilized, with higher capabilities and closer to the edge of the network, e.g., smart devices. Through a real deployment of our SensorClone in the wild, we identify that virtual devices can be used for two purposes, 1) to reduce the energy consumption of physical devices by duty cycling their service provisioning between the physical device and the virtual representation hosted in the cloud, and 2) to scale IoT services at the edge of the network by harnessing temporal periods of underutilization of smart devices. To evaluate our framework, we present a use case of a virtual sensor created from an IoT service of temperature. From our results, we verify that it is possible to achieve unlimited availability up to 90% and substantial power efficiency under acceptable levels of quality of service. Our work makes contributions towards improving IoT scalability and resilience by using virtual devices.

Online crowdsourcing enables the distribution of work to a global labor force as small and often repetitive tasks. Recently, situated crowdsourcing has emerged as a complementary enabler to elicit labor in specific locations and from specific crowds. Teamwork in online crowdsourcing has been recently shown to increase the quality of output, but teamwork in situated crowdsourcing remains unexplored. We set out to fill this gap. We present a generic crowdsourcing platform that supports situated teamwork and provide experiences from a laboratory study that focused on comparing traditional online crowdsourcing to situated team-based crowdsourcing. We built a crowdsourcing desk that hosts three networked terminal displays. The displays run our custom team-driven crowdsourcing platform that was used to investigate collocated crowdsourcing in small teams. In addition to analyzing quantitative data, we provide findings based on questionnaires, interviews, and observations. We highlight 1) emerging differences between traditional and collocated crowdsourcing, 2) the collaboration strategies that teams exhibited in collocated crowdsourcing, and 3) that a priori team familiarity does not significantly affect collocated interaction in crowdsourcing. The approach we introduce is a novel multi-display crowdsourcing setup that supports collocated labor teams and along with the reported study makes specific contributions to situated crowdsourcing research.

Computational offloading can improve user experience of mobile apps through improved responsiveness and reduced energy footprint. A fundamental challenge in offloading is to distinguish situations where offloading is beneficial from those where it is counterproductive. Currently, offloading decisions are predominantly based on profiling performed on individual devices. While significant gains have been shown in benchmarks, these gains rarely translate to real-world use due to the complexity of contexts and parameters that affect offloading. We contribute by proposing crowdsensed evidence traces as a novel mechanism for improving the performance of offloading systems. Instead of limiting to profiling individual devices, crowdsensing enables characterizing execution contexts across a community of users, providing better generalisation and coverage of contexts. We demonstrate the feasibility of using crowdsensing to characterize offloading contexts through an analysis of two crowdsensing datasets. Motivated by our results, we present the design and development of the EMCO toolkit and platform as a novel solution for computational offloading. Experiments carried out on a testbed deployment in Amazon EC2 Ireland demonstrate that EMCO can consistently accelerate app execution while at the same time reduce energy footprint. We also demonstrate that EMCO provides better scalability than current cloud platforms, being able to serve a larger number of clients without variations in performance. Ourframework, use cases, and tools are available as open source from GitHub.

Near Infrared Spectroscopy (NIRS) is a sensing technique in which near infrared light is transmitted into a sample, followed by light absorbance measurements at various wavelengths. This technique enables the inference of the inner chemical composition of the scanned sample, and therefore can be used to identify or classify objects. In this paper, we describe how to facilitate the use of NIRS by non- expert users in everyday settings. Our work highlights the key challenges of placing NIRS devices in the hands of non-experts. We develop a system to mitigate these challenges, and evaluate it in a user study. We show how NIRS technology can be successfully utilised by untrained users in an unsupervised manner through a special enclosure and an accompanying smartphone app. Finally, we discuss potential future developments of commoditised NIRS.

We seek to quantify smartwatch use, and establish differences and similarities to smartphone use. Our analysis considers use traces from 307 users that include over 2.8 million notifications and 800,000 screen usage events, and we compare our findings to previous work that quantifies smartphone use. The results show that smartwatches are used more briefly and more frequently throughout the day, with half the sessions lasting less than 5 seconds. Interaction with notifications is similar across both types of devices, both in terms of response times and preferred application types. We also analyse the differences between our smartwatch dataset and a dataset aggregated from four previously conducted smartphone studies. The similarities and differences between smartwatch and smartphone use suggest effect on usage that go beyond differences in form factor.

The quality of service of a mobile application is critical to ensure user satisfaction. Techniques have been proposed to accomplish adaptation of quality of service dynamically. However, there is still a limited understanding about how to provide a utility model for code execution. One key challenge is modeling the level of quality in the code execution that can be provisioned by the cloud. Since the allocation of cloud resources has a cost, it is important to optimize cloud usage. We propose a software-defined networking approach that allows modeling and controlling code acceleration of a mobile application deployed across multiple type of devices. By segregating the computational requirements of the mobile application into groups, we were able to define the acceleration needed by each group of devices. As the computational requirements of a device can change across time, a mobile device can be re-assigned to another group based on demand. Our SDN approach implements a model that allows the system to predict workload based on acceleration groups. Evaluating our system in a real testbed showed that it is possible to predict workload and allocate optimal resources to handle that workload with 87.5% accuracy.

Previous work suggests that Quantified-Self applications can retain long-term usage with motivational methods. These methods often require intermittent attention requests with manual data input. This may cause unnecessary burden to the user, leading to annoyance, frustration and possible application abandonment. We designed a novel method that uses on-screen alert dialogs to transform recurrent smartphone usage sessions into moments of data contributions and evaluate how accurately machine learning can reduce unintended interruptions. We collected sensor data from 48 participants during a 4-week long deployment and analysed how personal device usage can be considered in scheduling data inputs. We show that up to 81.7% of user interactions with the alert dialogs can be accurately predicted using user clusters, and up to 75.5% of unintended interruptions can be prevented and rescheduled. Our approach can be leveraged by applications that require self-reports on a frequent basis and may provide a better longitudinal QS experience.

Here we propose an interactive system to augment creative design thinking using networks of concepts in a virtual reality environment. We discuss how to augment the human capacity to be creative through dynamic suggestions providing new and original ideas, based on specific semantic network characteristics. We outline directions to explore the structures of the concept network and their connection to creative concept generation. It is expected that augmented creative thinking will allow the user to have more original ideas and thus be more innovative.

We present a study that quantifies the effect of cold temperature on smartphone input performance, particularly on tapping tasks. Our results show that smartphone input performance decreases when completing tapping tasks in cold temperatures. We show that colder temperature is associated with lower throughput and less accurate performance when using the phone in both one-handed and two-handed operations. We also demonstrate that colder temperature is related to higher error rate when using the phone in one-handed operation only, but not two-handed. Finally, we identify a number of design recommendations from the literature that can be considered as a countermeasure to poorer smartphone input performance in completing tapping tasks in cold temperature.

It is becoming increasingly easy for researchers to develop context-aware applications for smartphones. A perennial challenge, however, is to convince a large number of people to install them and donate contextual data for scientific purposes. Our empirical study seeks to address this challenge by investigating how people's perception and attitude affect their willingness to donate context data to researchers and quantifies the effects of social signals on donation action-taking. Our findings indicate that the perceived need for donation and perceived organization reputation are key determinants in deciding whether to donate: people with altruistic personality do not necessarily donate if they cannot see the need to take an action. Furthermore, we provide evidence that even if people indicate a willingness to donate, they are hesitant to take action towards donating data unless catalysts like social signals (hints about the actions of others) are present.

Mobile offloading is a promising technique to aid the constrained resources of a mobile device. By offloading a computational task, a device can save energy and increase the performance of the mobile applications. Unfortunately, in existing offloading systems, the opportunistic moments to offload a task are often sporadic and short-lived. We overcome this problem by proposing a social-aware hybrid offloading system (HyMobi), which increases the spectrum of offloading opportunities. As a mobile device is always co-located to at least one source of network infrastructure throughout of the day, by merging cloudlet, device-to-device and remote cloud offloading, we increase the availability of offloading support. Integrating these systems is not trivial. In order to keep such coupling, a strong social catalyst is required to foster user's participation and collaboration. Thus, we equip our system with an incentive mechanism based on credit and reputation, which exploits users’ social aspects to create offload communities. We evaluate our system under controlled and in-the-wild scenarios. With credit, it is possible for a device to create opportunistic moments based on user's present need. As a result, we extended the widely used opportunistic model with a long-term perspective that significantly improves the offloading process and encourages unsupervised offloading adoption in the wild.

We present a crowdsourcing methodology to elicit highly structured knowledge for arbitrary questions. The method elicits potential answers (“options”), criteria against which those options should be evaluated, and a ranking of the top “options.” Our study shows that situated crowdsourcing markets can reliably elicit/moderate knowledge to generate a ranking of options based on different criteria that correlate with established online platforms. Our evaluation also shows that local crowds can generate knowledge that is missing from online platforms and on how a local crowd perceives a certain issue. Finally, we discuss the benefits and challenges of eliciting structured knowledge from local crowds.

Researchers who analyse smartphone usage logs often make the assumption that users who lock and unlock their phone for brief periods of time (e.g., less than a minute) are continuing the same "session" of interaction. However, this assumption is not empirically validated, and in fact different studies apply different arbitrary thresholds in their analysis. To validate this assumption, we conducted a field study where we collected user-labelled activity data through ESM and sensor logging. Our results indicate that for the majority of instances where users return to their smartphone, i.e., unlock their device, they in fact begin a new session as opposed to continuing a previous one. Our findings suggest that the commonly used approach of ignoring brief standby periods is not reliable, but optimisation is possible. We therefore propose various metrics related to usage sessions and evaluate various machine learning approaches to classify gaps in usage.

Research claims that users value the battery life of their smartphones, but no study to date has attempted to quantify battery value and how this value changes according to users' current context and needs. Previous work has quantified the monetary value that smartphone users place on their data (e.g., location), but not on battery life. Here we present a field study and methodology for systematically measuring the monetary value of smartphone battery life, using a reverse second-price sealed-bid auction protocol. Our results show that the prices for the first and last 10% battery segments differ substantially. Our findings also quantify the tradeoffs that users consider in relation to battery, and provide a monetary model that can be used to measure the value of apps and enable fair ad-hoc sharing of smartphone resources.

We present the development and evaluation of a situated crowdsourcing mechanism that estimates queue length in real time. The system relies on public interactive kiosks to collect human estimations about their queue waiting time. The system has been designed as a standalone tool that can be retrospectively embedded in a variety of locations without interfacing with billing or customer systems. An initial study was conducted in order to determine whether people who just joined the queue would differ in their estimates from people who were at the front of the queue. We then present our system's evaluation in four different restaurants over 19 weekdays. Our analysis shows how our system is perceived by users, and we develop 2 ways to optimise the waiting time estimation: by correcting the estimations based on the position of the input mechanism, and by changing the sliding window considered inputs to provide better prediction. Our analysis shows that approximately 7% of restaurant customers provided estimations, but even so our system can provide predictions with up to 2 minute mean absolute error.

We develop a Markov state transition model of smartphone screen use. We collected use traces from real-world users during a 3-month naturalistic deployment via an app-store. These traces were used to develop an analytical model which can be used to probabilistically model or predict, at runtime, how a user interacts with their mobile phone, and for how long. Unlike classification-driven machine learning approaches, our analytical model can be interrogated under unlimited conditions, making it suitable for a wide range of applications including more realistic automated testing and improving operating system management of resources.

We develop and evaluate a data hiding method that enables smartphones to encrypt and embed sensitive information into carrier streams of sensor data. Our evaluation considers multiple handsets and a variety of data types, and we demonstrate that our method has a computational cost that allows real-time data hiding on smartphones with negligible distortion of the carrier stream. These characteristics make it suitable for smartphone applications involving privacy-sensitive data such as medical monitoring systems and digital forensics tools.

We evaluate the situational impairments caused by cold ambient temperature on fine-motor movement and vigilance during mobile interaction. For this purpose, we tested two mobile phone applications that measure fine motor skills and vigilance in controlled temperature settings. Our results show that cold adversely affected participants' fine-motor skills performance, but not vigilance. Based on our results we highlight the importance of correcting measurements when investigating performance of cognitive tasks to take into account the physical element of the tasks. Finally, we identify a number of design recommendations from literature that can mitigate the adverse effect of cold ambiance on interaction with mobile devices.

There is a limited amount of scavenging alternatives for smartphones. We assess the feasibility of using indoor light to extend smartphones' battery life. We build a prototype charger that demonstrates that indoor light scavenging is a practical method that can substantially extend battery life on smartphones. The results show that it is feasible and practical to extend battery life with this energy harvesting method. We finally discuss certain obstacles that need to be overcome, especially the redesign of operating systems to account for energy harvesting.

In information systems (IS) literature, there is ongoing debate as to whether the field has become fragmented and lost its identity in response to the rapid changes of the field. The paper contributes to this discussion by providing quantitative measurement of the fragmentation or cohesiveness level of the field. A co-word analysis approach aiding in visualization of the intellectual map of IS is applied through application of clustering analysis, network maps, strategic diagram techniques, and graph theory for a collection of 47,467 keywords from 9551 articles, published in 10 major IS journals and the proceedings of two leading IS conferences over a span of 20 years, 1993 through 2012. The study identified the popular, core, and bridging topics of IS research for the periods 1993–2002 and 2003–2012. Its results show that research topics and subfields underwent substantial change between those two periods and the field became more concrete and cohesive, increasing in density. Findings from this study suggest that the evolution of the research topics and themes in the IS field should be seen as part of the natural metabolism of the field, rather than a process of fragmentation or disintegration.

We present an empirical study that investigates crowdsourcing performance in a situated market. Unlike online markets, situated crowdsourcing markets consist of workers who become serendipitously available for work in a particular location and context. So far, the literature has lacked a systematic study of task performance and uptake in such markets under varying incentives. In a 3-week field study, we demonstrate that in a situated crowdsourcing market, task uptake and accuracy are generally comparable with online markets. We also show that increasing task rewards in situated crowdsourcing leads to increased task uptake but not accuracy, while decreasing task rewards leads to decreases in both task uptake and accuracy.

We present a revisitation analysis of smartphone use to investigate the question: do smartphones induce usage habits? We analysed three months of application launch logs from 165 users in naturalistic settings. Our analysis reveals distinct clusters of applications and users which share similar revisitation patterns. However, we show that much of smartphone usage on a macro-level is very similar to web browsing on desktops, and thus argue that smartphone usage is driven by innate service needs rather than technology characteristics. On the other hand, on a micro-level we identify unique characteristics in smartphone usage, and we present a rudimentary model that accounts for 92% in the variability of our smartphone use.

Smartphone users do not fully know what their apps do. For example, an applications' network usage and underlying security configuration is invisible to users. In this paper we introduce Securacy, a mobile app that explores users' privacy and security concerns with Android apps. Securacy takes a reactive, personalized approach, highlighting app permission settings that the user has previously stated are concerning, and provides feedback on the use of secure and insecure network communication for each app. We began our design of Securacy by conducting a literature review and in-depth interviews with 30 participants to understand their concerns. We used this knowledge to build Securacy and evaluated its use by another set of 218 anonymous participants who installed the application from the Google Play store. Our results show that access to address book information is by far the biggest privacy concern. Over half (56.4%) of the connections made by apps are insecure, and the destination of the majority of network traffic is North America, regardless of the location of the user. Our app provides unprecedented insight into Android applications' communications behavior globally, indicating that the majority of apps currently use insecure network connections.

In this paper we present Game of Words, a crowdsourcing game for public displays that allows the creation of a keyword dictionary to describe locations. It relies on crowdsourcing and gamification to identify, filter, and rank keywords based on their relevance to the location of the public display itself. We demonstrate that crowdsourcing on public displays can leverage users' knowledge of their environment, can work with a generic gaming task, and can be deployed on displays with multiple concurrent services. Our analysis shows that our approach has important benefits, such as the ability to identify undesired input, provide words of high semantic relevance, as well as a broader scope of keywords. Finally, our analysis also demonstrates that the chosen game design coped well with the challenges of this complex setting (i.e. public urban space) by disincentivising incorrect use of the system.

This study employs hierarchical cluster analysis, strategic diagrams and network analysis to map and visualize the intellectual landscape of the CHI conference on Human Computer Interaction through the use of co-word analysis. The study quantifies and describes the thematic evolution of the field based on a total of 3152 CHI articles and their associated 16035 keywords published between 1994 and 2013. The analysis is conducted for two time periods (1994-2003, 2004-2013) and a comparison between them highlights the underlying trends in our community. More significantly, this study identifies the evolution of major themes in the discipline, and highlights individual topics as popular, core, or backbone research topics within HCI.

Projective tests are personality tests that reveal individuals' emotions (e.g., Rorschach inkblot test). Unlike direct question-based tests, projective tests rely on ambiguous stimuli to evoke responses from individuals. In this paper we develop one such test, designed to be delivered automatically, anonymously and to a large community through public displays. Our work makes a number of contributions. First, we develop and validate in controlled conditions a quantitative projective test that can reveal emotions. Second, we demonstrate that this test can be deployed on a large scale longitudinally: we present a four-week deployment in our university's public spaces where 1431 tests were completed anonymously by passers-by. Third, our results reveal strong diurnal rhythms of emotion consistent with results we obtained independently using the Day Reconstruction Method (DRM), literature on affect, well-being, and our understanding of our university's daily routine.

We present Pulse, a wireless magnetic communication protocol for smartphones. Pulse is designed for off-the-shelf Android smartphones with magnetometers, and encodes data in magnetic fields. We present the design and evaluation of Pulse in various conditions (e.g., different voltages, number of transfer channels). The system provides security due to its short range (~1 cm), it can reach a speed of up to 44 bits per second, and it is possible to run it on most mobile phones with a magnetometer. We present our evaluation and discuss practical use cases where Pulse can be used today.

The rapid growth of the Ubicomp field has recently raised concerns regarding its identity. These concerns have been compounded by the fact that there exists a lack of empirical evidence on how the field has evolved until today. In this study we applied co-word analysis to examine the status of Ubicomp research. We constructed the intellectual map of the field as reflected by 6858 keywords extracted from 1636 papers published in the HUC, UbiComp and Pervasive conferences during 1999--2013. Based on the results of a correspondence analysis we identify two major periods in the whole corpus: 1999--2007 and 2008--2013. We then examine the evolution of the field by applying graph theory and social network analysis methods to each period. We found that Ubicomp is increasingly focusing on mobile devices, and has in fact become more cohesive in the past 15 years. Our findings refute the assertion that Ubicomp research is now suffering an identity crisis.

Research suggests smartphone users face 'application overload', but literature lacks an in-depth investigation of how users manage their time on smartphones. In a 3-week study we collected smartphone application usage patterns from 21 participants to study how they manage their time interacting with the device. We identified events we term application micro-usage: brief bursts of interaction with applications. While this practice has been reported before, it has not been investigated in terms of the context in which it occurs (e.g., location, time, trigger and social context). In a 2-week follow-up study with 15 participants, we captured participants? context while micro-using, with a mobile experience sampling method (ESM) and weekly interviews. Our results show that about approximately 40% of application launches last less than 15 seconds and happen most frequently when the user is at home and alone. We further discuss the context, taxonomy and implications of application micro-usage in our field. We conclude with a brief reflection on the relevance of short-term interaction observations for other domains beyond mobile phones.

In an increasingly connected world, users access personal or shared data, stored "in the cloud" (e.g., Dropbox, Skydrive, iCloud) with multiple devices. Despite the popularity of cloud storage services, little work has focused on investigating cloud storage users' Quality of Experience (QoE), in particular on mobile devices. Moreover, it is not clear how users' context might affect QoE. We conducted an online survey with 349 cloud service users to gain insight into their usage and affordances. In a 2-week follow-up study, we monitored mobile cloud service usage on tablets and smartphones, in real-time using a mobile-based Experience Sampling Method (ESM) questionnaire. We collected 156 responses on in-situ context of use for Dropbox on mobile devices. We provide insights for future QoE-aware cloud services by highlighting the most important mobile contextual factors (e.g., connectivity, location, social, device), and how they affect users' experiences while using such services on their mobile devices.

Research is increasingly highlighting the potential for situated crowdsourcing to overcome some crucial limitations of online crowdsourcing. However, it remains unclear whether a situated crowdsourcing market can be sustained, and whether worker supply responds to price-setting in such a market. Our work is the first to systematically investigate workers' behaviour and response to economic incentives in a situated crowdsourcing market. We show that the market-based model is a sustainable approach to recruiting workers and obtaining situated crowdsourcing contributions. We also show that the price mechanism is a very effective tool for adjusting the supply of labour in a situated crowdsourcing market. Our work advances the body of work investigating situated crowdsourcing.

Transitioning from bespoke single-purpose displays to multipurpose public interactive displays entails a number of challenges. One challenge is the development of usable mechanisms that allow users to explore the functionality and services on such displays. This article presents a field trial that employs AutoCardSorter, a tool that uses semantic similarity and clustering algorithms, to automatically group the available applications of a public interactive display into categories based on the developer-provided descriptions of each application. The results demonstrate that the grouping generated by AutoCardSorter improved both performance and self-reported usability measures compared to practitioners' existing grouping. In addition, the study investigated the interplay between grouping and interaction modality (i.e., public display vs. desktop). Results tend to support that grouping affects more the user experience with a multipurpose interactive display, but findings were insignificant. This work provides a way for public displays to dynamically update their offered services without sacrificing usability.

Although advances in technology now enable people to communicate ‘anytime, anyplace’, it is not clear how citizens can be motivated to actually do so. This paper evaluates the impact of three principles of psychological empowerment, namely perceived self-efficacy, sense of community and causal importance, on public transport passengers’ motivation to report issues and complaints while on the move. A week-long study with 65 participants revealed that self-efficacy and causal importance increased participation in short bursts and increased perceptions of service quality over longer periods. Finally, we discuss the implications of these findings for citizen participation projects and reflect on design opportunities for mobile technologies that motivate citizen participation.

Safeguarding personally identifiable information (PII) is crucial because such information is increasingly used to engineer privacy attacks, identity thefts and security breaches. But is it likely that individuals may choose to just share this information with strangers? This study examines how reciprocation can lead to the disclosure of PII between strangers in online social networking. We demonstrate that the widespread use of public, one-to-many, communication channels such as ‘wall posts’ and profile pages in online social networks poses an exception to the assumption that reciprocation happens on one-to-one channels. We find that individuals not only reciprocate and share PII when the disclosure of such information is private and directed towards them by a stranger, but also when the stranger shares this information through a public channel that is not directed towards anyone in particular. Implications for privacy and design are discussed.

In this study, we demonstrate how analysis of users’ social network structure—a topic that has remained until recently inconspicuous within Human-Computer Interaction (HCI) research on social systems—can contribute to our understanding of Social Networking Services (SNS) effect on users. Despite a consensus that SNS enhance people's social capital, prior studies on SNS have provided inconsistent evidence on this process. In a multipronged study, we analyze personality, social capital, and Facebook data from a cohort of participants to model the extent to which one's SNS reflects aspects of his or personality and affects his bridging social capital. Our empirically validated model shows that empathy and conscientiousness influence the structural holes in one's social network, which in turn affects bridging social capital. These findings highlight the importance of network structure as an intermediary between one's personality and the social benefits one reaps from using SNS. Our work demonstrates how the implicit structural information embedded in users’ social networks can provide key insights into users’ personality and social capital.

Awareness campaigns aiming to highlight the accessibility challenges affecting people with disabilities face an important challenge. They often describe the environmental features that pose accessibility barriers out of context, and as a result public cannot relate to the problems at hand. In this paper we demonstrate that contextual cues can enhance people's perception and understanding of accessibility. We describe a two-week study where our participants submitted reports of inaccessible spots all over the city through a web application. Using a 2x2 factorial design we contrast the impact of two types of contextual cues, visual cues (i.e., displaying a picture of the inaccessible spot) and location cues (i.e., ability to zoom-in the exact location). We measure participants' perceptions of accessibility and how they are challenged to consider their own limitations and barriers that may also affect themselves in certain circumstances. Our results suggest that visual cues led to a bigger sense of urgency while also improving participants' attitude towards disability.

Narrowcasting refers to the targeted segmentation of media dissemination, and has been proposed as a counterpart to broadcasting. We present an explorative study that evaluates narrowcasting as an approach to sharing in online social media. We test a narrowcasting prototype for Facebook with 54 participants over a four-week period. We outline the various strategies that participants used to appropriate narrowcasting, and report on participants' use and perceptions. We also report on the effects of default sharing options and gender on sharing behavior. Our work provides implications for online sharing, suggesting that narrowcasting is an effective strategy for online social platforms.

In this paper we adopt a network science approach to investigate empathy and its implications for online social networks. We demonstrate that empathy is closely linked to social capital - the findings suggest that individuals higher on cognitive empathic skill are overall likely to report both higher bridging and higher bonding social capital. On the other hand, attributes of network structure around the individual, quantified through networks analysis metrics, were related to cognitive empathy. Further, an examination of the interplay between network structure, social capital and empathy suggests that empathy facilitates the relation between network structure and social capital previously reported in literature. We discuss the implications of our findings for the understanding of empathy in the context of online social networks and for the design of these systems.

Most studies take for granted the critical first steps that prelude interaction with a public display: awareness of the interactive affordances of the display, and enticement to interact. In this paper we investigate mechanisms for enticing interaction on public displays, and study the effectiveness of visual signals in overcoming the 'first click' problem. We combined 3 atomic visual elements (color/greyscale, animation/static, and icon/text) to form 8 visual signals that were deployed on 8 interactive public displays on a university campus for 8 days. Our findings show that text is more effective in enticing interaction than icons, color more than greyscale, and static signals are more effective than animated. Further, we identify gender differences in the effectiveness of these signals. Finally, we identify a behavior termed "display avoidance" that people exhibit with interactive public displays.

Mobile phone user interfaces typically show an icon to indicate remaining battery, but not the amount of time the device can be used for, often forcing users to make faulty estimates and predictions about battery life. Here we report on two studies that capture users' experiences with a user-centered battery interface design. In Study 1, we analyze 12 participants' use of mobile phones, demonstrating that mobile phone users do not know how or what to do to extend their mobile's battery life. We further identify the information they rely on to assess battery life. In Study 2, we use this information to design, prototype and evaluate an interactive battery interface (IBI) with another 22 participants. Our findings describe how users perceive battery life and how we used their mental models of mobile phone batteries to create IBI. Lastly, we report on the users' experiences and IBI's effect on battery lifetime, showing gains of approximately 27% over the course of a day.

This study is the first attempt to investigate altruistic use of interactive public displays in natural usage settings as a crowdsourcing mechanism. We test a non-paid crowdsourcing service on public displays with eight different motivation settings and analyse users' behavioural patterns and crowdsourcing performance (e.g., accuracy, time spent, tasks completed). The results show that altruistic use, such as for crowdsourcing, is feasible on public displays, and through the controlled use of motivational design and validation check mechanisms, performance can be improved. The results shed insights on three research challenges in the field: i) how does crowdsourcing performance on public displays compare to that of online crowdsourcing, ii) how to improve the quality of feedback collected from public displays which tends to be noisy, and iii) identify users' behavioural patterns towards crowdsourcing on public displays in natural usage settings.

Better tools for content-based access of video are needed to improve access to time-continuous video data. Particularly information about linear TV broadcast programs has been available in a form limited to program guides that provide short manually described overviews of the program content. Recent development in digitalization of TV broadcasting and emergence of web-based services for catch-up and on-demand viewing bring out new possibilities to access data. In this paper we introduce our data mining system and accompanying services for summarizing Finnish DVB broadcast streams from seven national channels. We describe how data mining of novelty concepts can be extracted from DVB subtitles to augment web-based "Catch-Up TV Guide" and "Novelty Cloud" TV services. Furthermore, our system allows accessing media fragments as Picture Quotes via generated word lists and provides content-based recommendations to find new programs that have content similar to the user selected programs. Our index consists of over 180 000 programs that are used to recommend relevant programs. The service has been under development and available online since 2010. It has registered over 5000 user sessions.

Thanks to the rapid development of mobile technologies, smartphones allow people to be reachable anywhere and anytime. In addition to the benefits for end users, researchers and developers can also benefit from the powerful devices that participants potentially carry on a daily basis. This minitrack workshop brings together researchers with an interest on using mobile devices as instruments to collect data and conduct mobile user studies, with a focus on understanding human-behavior, routines and gathering context.

The proliferation of social networking sites and mobile technology allows us to check on our friends and family, follow what experts in our field think, or simply 'check-in' online. While in many ways advantageous, the ability to be constantly connected is significantly affecting our offline interaction behavior. People sharing a table today might ignore each other for stretches at a time in order to interact with far-away friends through mobile technology instead. The goal of this workshop is to examine how we can build technologies that promote offline interactions. We plan to discuss how offline interactions can be spurred within different social groups and different settings through currently available devices and technologies. We also plan to explore how such technologies can be built and used for different types of offline engagement (e.g., playful vs. serious). The workshop aims to establish a community interested in computer mediated offline interaction.

Open mobile platforms such as Android currently suffer from the existence of multiple versions, each with its own peculiarities. This makes the comprehensive testing of interactive applications challenging. In this paper we present Testdroid, an online platform for conducting scripted user interface tests on a variety of physical Android handsets. Testdroid allows developers and researchers to record test scripts, which along with their application are automatically executed on a variety of handsets in parallel. The platform reports the outcome of these tests, enabling developers and researchers to quickly identify platforms where their systems may crash or fail. At the same time the platform allows us to identify more broadly the various problems associated with each handset, as well as frequent programming mistakes.

We report two one-month-long field trials where Bluetooth access points deployed around Oulu, Finland, were employed to attempt to push unsolicited multimedia marketing messages to bypassing mobile devices that had their Bluetooth on and visible. The logs involving ~65000 unique discovered devices of real users show that only 0.12% of the ~650000 transmission attempts were successful. On average, 1.1% of the devices received the message and 3.3% of the owners of these devices signed up to the marketing campaign. These statistics characterize the efficiency of 'carpet bombing' type of proximity marketing realized with the current Bluetooth technology without any support mechanisms.

This paper presents a study that aims to answer two important questions related to targeted location-sharing privacy attacks: (1) given a group of users and their social graph, is it possible to predict which among them is likely to reveal most about their whereabouts, and (2) given a user, is it possible to predict which among her friends knows most about her whereabouts. To answer these questions we analyse the privacy policies of users of a real-time location sharing application, in which users actively shared their location with their contacts. The results show that users who are central to their network are more likely to reveal most about their whereabouts. Furthermore, we show that the friend most likely to know the whereabouts of a specific individual is the one with most common contacts and/or greatest number of contacts.

This paper introduces the UBI Challenge that challenged the global R&D community to design, implement, deploy and evaluate novel applications and services in real world setting atop an open urban computing testbed. The paper first provides a procedural description of the UBI Challenge and then discusses the outcome so far with a special focus on the various issues introduced by the real world setting.

Tangible user interfaces are portals to digital information. In the future, securing access to such material will be an important concern. This paper describes the design, implementation and evaluation of a PIN entry system based on audio or haptic cues that is suitable for integration into such physical systems. The current implementation links movements on a mobile phone touch screen with the display of non-visual cues; selection of a sequence of these cues composes a password. Studies reveal the validity of this approach in terms of task times and error rates that improve over prior art. In sum, this paper demonstrates the potential of non-visual PINs as a mechanism for securing access to a range of systems, ultimately incorporating mobile, ubiquitous or tangible interfaces.

Research on tangible interaction and digital haptics has rarely intertwined, despite the natural relationship between physicality and touch. This paper addresses this relatively unexplored domain by presenting the Haptic Wheel, a freestanding single-axis rotational controller incorporating vibro-tactile cues. In addition to describing the hardware and implementation, the paper discusses the potential application of the system for eyes-free interaction, password entry and as an active puck on a tabletop system. The paper suggests that systems with active haptic feedback have unexploited potential as tools for tangible interaction.

This paper presents a multi-pronged study of users' location-sharing practices in the context of online social networks. The contribution of this study is two-fold: first it presents a series of insights relating to location-sharing practices, and second it highlights the use of third-person scenarios as a useful method for eliciting privacy concerns and potentially educating users.

Moving human-computer interaction off the desktop and into our cities requires new approaches to understanding people and technologies in the built environment. We approach the city as a system, with human, physical and digital components and behaviours. In creating effective and usable urban pervasive computing systems, we need to take into account the patterns of movement and encounter amongst people, locations, and mobile and fixed devices in the city. Advances in mobile and wireless communications have enabled us to detect and record the presence and movement of devices through cities. This article makes a number of methodological and empirical contributions. We present a toolkit of algorithms and visualization techniques that we have developed to model and make sense of spatial and temporal patterns of mobility, presence, and encounter. Applying this toolkit, we provide an analysis of urban Bluetooth data based on a longitudinal dataset containing millions of records associated with more than 70000 unique devices in the city of Bath, UK. Through a novel application of established complex network analysis techniques, we demonstrate a significant finding on the relationship between temporal factors and network structure. Finally, we suggest how our understanding and exploitation of these data may begin to inform the design and use of urban pervasive systems.

This paper examines a visual design element unique to situated, hot-spot style, services: locativeness. This is the extent to which the media representing a service relates to its immediate physical environment. This paper explores the effect of locativeness on trust with two studies assessing user attitudes in depth. The first is an implicit, or preconscious, test and the second an explicit test based on voiced value judgments. To provide a richer context, the second study contrasts locativeness with other traditional aspects of design: branding and quality. The results indicate users have a strong implicit association between locative images and trust, and that this is partially reflected in their explicit choices. This is an important interface aspect that designers should consider in order to create trustworthy situated services.

Mass casualty incidents generate a sequence of response events from the emergency services, requiring the allocation and use of resources in a timely fashion. In this paper we describe a pervasive system that helps emergency services optimize their efficiency and coordination. The system emulates a multiple casualty emergency response environment in which contextual information, retrieved from victims' wearable or mobile devices, guides early assessments on the health condition of the affected population. Additionally, we analyze the behavior of our system under different conditions and derive the necessary parameters for achieving accurate estimations. Our main contribution is the enhancement of the existing emergency response process for mass casualty incidents through the use of pervasive computing technology.

Journal Article The social implications of emerging technologies Get access Vassilis Kostakos, Vassilis Kostakos * * Corresponding authors Search for other works by this author on: Oxford Academic Google Scholar Eamonn O'Neill, Eamonn O'Neill Search for other works by this author on: Oxford Academic Google Scholar Linda Little, Linda Little * * Corresponding authors Search for other works by this author on: Oxford Academic Google Scholar Elizabeth Sillence Elizabeth Sillence Search for other works by this author on: Oxford Academic Google Scholar Interacting with Computers, Volume 17, Issue 5, September 2005, Pages 475–483, https://doi.org/10.1016/j.intcom.2005.03.001 Published: 01 September 2005

In this paper we discuss a real world problem encountered during recent fieldwork: that of providing information in public settings when the information has both public and private components. We draw on our ethnographic studies in the waiting area of a busy hospital Emergency department. Despite evidence that lack of information can lead to stress, problem behaviours and poor levels of satisfaction with treatment, little information was made available to patients. We review the types of information needed and propose how the theoretical concepts of public, social and private information spheres relate to public spaces such as the Emergency department waiting area. We argue how the further theoretical concept of interaction spaces may be used in conjunction with these information spheres to inform interaction design for public settings.