Extensive research has examined presence and basic psychological needs (drawing on Self-Determination Theory) in digital media. While prior work offers hints of potential connections, we lack a systematic account of whether and how distinct presence dimensions map onto the basic needs of autonomy, competence, and relatedness. We surveyed 301 social VR users and analyzed using Structural Equation Modeling. Results show that social presence predicts all three needs, while self-presence predicts competence and relatedness, and spatial presence shows no direct or moderating effects. Gender and age moderated these relationships: women benefited more from social presence for autonomy and relatedness, men from self- and spatial presence for competence and autonomy, and younger users showed stronger associations between social presence and relatedness, and between self-presence and autonomy. These findings position presence as a motivational mechanism shaped by demographic factors. The results offer theoretical insights and practical implications for designing inclusive, need-supportive multiuser VR environments.

The sense of presence is central to immersive experiences in Virtual Reality (VR), and particularly salient in socially rich platforms like social VR. While prior studies have explored various aspects related to presence, less is known about how ongoing usage behaviors shape presence in everyday engagement. To address this gap, we examine whether usage intensity, captured through frequency of use, session duration, and years of VR experience, predicts presence in social VR. A survey of 295 users assessed overall, social, spatial, and self-presence using validated scales. Results show that both frequency and duration consistently predict higher presence across all dimensions, with interaction effects indicating that frequent and extended sessions synergistically amplify the experience of “being there.” These effects were stable across age and gender. Our findings extend presence research beyond the laboratory by identifying behavioral predictors in social VR and offer insights for building inclusive environments that reliably foster presence.

People face challenges with complex search tasks daily; chat-based large language models have the potential to assist search by helping with query formulation and content navigation, for instance. However, there is a lack of user research on integrating AI chat and traditional search to enhance Information Retrieval (IR) for the general public. This study explores how younger (25 participants) and older (22 participants) adults utilize these two platforms to retrieve information. Through two IR tasks, quality- and quantity-oriented tasks, focused on gathering evidence related to various countries’ CO 2 emissions, younger users spent more time, issued more queries, and explored more external web pages than older adults in both tasks. In contrast, despite reporting less familiarity with AI chat, older adults relied more on chat to complete their tasks. They submitted longer queries for the quality task and reached the first query faster for the quantity task than their younger counterparts, reflecting a top-down IR strategy. In the end, task outcomes were similar for the two groups, indicating AI chat’s capability to support IR. Drawing from our study results and relevant literature, we propose five design requirements to harness the synergic value of search and AI chat in supporting IR.

In social Virtual Reality (VR), particularly within VRChat, a significant group of users often referred to as "mutes" refrain from voice communication. This study analyzes 4212 discussion entries, including both original submissions and comments, from the r/VRchat subreddit to explore the experiences and reasons behind this practice. Our findings indicate that muteness is an integral aspect of social VR culture, yet mute users face challenges, including exposure to abusive behaviors and communication barriers in a fast-paced environment. Factors of social VR like harassment, heightened social anxiety from the immersive presence, and the complexities of identity management can discourage voice communication, leading many to adopt "muteness" as a response. This behavior can be seen within the broader context of social disability, challenging normative communication assumptions. We highlight the risks of generalizing marginalized communities and emphasize the need for further research to address and support the unique needs of these groups in social VR spaces.

Extensive research has focused on community-based moderation involving selected or volunteer moderators. The "votekick" system represents a democratized approach allowing all users to participate in moderation. Despite its widespread use in online gaming and social VR platforms, votekicking remains underexplored. This research studies how users use and perceive votekicking in VRChat, a leading social VR platform. Through thematic analysis of discussions from the Reddit community r/VRChat, our findings reveal that votekicking serves to cope with misconduct and enforce group-specific rules, but it also perpetuates toxicity such as materializing community-level biases. While praised for its immediacy and clear messaging against unacceptable behavior, votekicking's effectiveness is hindered by its reactive nature, consensus challenges, and decision-making complexities. This research contributes to broader discussions on the limitations and advantages of direct community involvement in moderation and suggests practical design improvements to address the challenges associated with votekicking.

LLM-based embodied agents are recently emerging in VR, supporting various scenarios such as pedagogical assistants, virtual companions, and NPCs for games. These agents hold potential to enhance user interactions but require thoughtful design to cater diverse user needs and contexts. We present an architecture that leverages different LLM modules to enable conversational interactions with an embodied agent in multi-user VR. Our system’s primary goal is to facilitate immersive tailoring through conversational input, allowing users to dynamically adjust an agent’s behavior and properties (e.g., role, personality, and appearance), directly within the virtual space, rather than during development or via separate interfaces. We evaluate the system’s performance, measuring latency during tailoring tasks, and share insights from a six-week study involving five users exploring various scenarios. While the approach shows promise, challenges remain, including reducing latency in the speech-to-text-to-speech pipeline and addressing the black-box limitations of LLMs, highlighting areas for future research.

In social Virtual Reality (VR) environments, the significant trend of 'mirror dwellers,' users who often use virtual mirrors to engage with their avatars, has emerged. This study examines discussions from r/VRchat to explore the reasons for this behavior and how it is perceived within the broader community. Our findings highlight the critical role of mirrors in compensating for the sensory limitations of VR, particularly the lack of physical feedback. Users often turn to mirrors to view parts of their avatar that are not accessible from a first-person perspective. Additionally, our research uncovers that a limited Field-Of-View (FOV) hinders the development of a strong connection between users and their avatars, further driving the need for mirrors. However, while using mirrors to mitigate FOV and physical feedback limitations can be helpful, it may also disrupt social interaction in VR environments, as the excessive reliance on mirrors can hinder the social experience in VR for others. This research deepens our understanding of user behavior in social VR and provides insights that could guide future design improvements to enrich the overall user experience.

This study explores the use of 'blocking' as a safety tool in social VR platforms, a feature widely implemented to protect users from harassment. Despite its prevalence, little research investigates the reasons users block others and how they perceive the feature.To address this gap, we analyzed discussions from r/VRChat, r/AltspaceVR, r/HorizonWorld, and r/RecRoom, using thematic analysis. Our findings reveal that users block others for various reasons, extending beyond mere disruption avoidance to include personal preferences of avatars and social management such as blocking a friend for a disagreement. Furthermore, the study highlights the empowering roles and the limitations of blocking as a solution to harassment. The study contributes to understanding user behavior in social VR and offers insights for developing more effective safety tools in these immersive environments.

Drinking in social VR has become popular, yet little is known about how users perceive and experience alcohol consumption while immersed in virtual spaces with others, as well as its potential harm and negative effects on their offline and online lives. To better understand this emerging phenomenon from the perspective of both drinkers and non-drinkers, we analyzed public discussions from the r/VRchat online community on users’ perceptions, and experiences with alcohol consumption in social VR. Heavy drinking is prevalent. We find that VR drinkers feel less intoxicated, which makes them drink more without being aware of it. Anti-cybersickness designs may affect users’ perception of vertigo, even if the vertigo is not caused by VR. We discuss how affordances that support meaningful activities (i.e., sense of presence, embodiment, and social interactions) exacerbate alcohol abuse. We propose implications for the design of safer social VR experiences for both drinkers and non-drinkers.

To compare and select machine learning models, relying on performance measures alone may not always be sufficient. This is particularly the case where different subsets, features, and predicted results may vary in importance relative to the task at hand. Explanation and visualization techniques are required to support model sensemaking and informed decision-making. However, a review shows that existing systems are mostly designed for model developers and not evaluated with target users in their effectiveness. To address this issue, this research proposes an interactive visualization, VMS (Visualization for Model Sensemaking and Selection), for users of the model to compare and select predictive models. VMS integrates performance-, instance-, and feature-level analysis to evaluate models from multiple angles. Particularly, a feature view integrating the value and contribution of hundreds of features supports model comparison on local and global scales. We exemplified VMS for comparing models predicting patients’ hospital length of stay through time-series health records and evaluated the prototype with 16 participants from the medical field. Results reveal evidence that VMS supports users to rationalize models in multiple ways and enables users to select the optimal models with a small sample size. User feedback suggests future directions on incorporating domain knowledge in model training, such as for different patient groups considering different sets of features as important.

With the advent of consumer-grade electroencephalography (EEG) and virtual reality (VR) devices, the use of human cognitive processes directly as means for user-adapted interaction in immersive virtual environments has become increasingly relevant. In this study (N = 43), we investigate electroencephalography-based neurofeedback interaction in virtual reality (VR). Particularly, we investigate this phenomenon in the context of meditation which enables the studying of cognitive processes (attention, sense of presence, meditation depth) when using an immersive interface that is adaptive based on neural responses. A prototype virtual reality environment was built that employs head-mounted display (HMD), and the neurofeedback functionality guided the user to increase the neural indices of meditation-related concentration and relaxation. The observed findings provide evidence for the effectiveness of neurofeedback functionality and VR in evoking desired types of neural activation and subjective experiences.

Abstract Social Virtual Reality (VR) is growing in popularity and has drawn the attention of HCI academics. Social VR experiences harassment just like other online environments. The Trust System (TS) in VRChat, one of the most prominent social VR platforms, is designed to measure and indicate users’ trustworthiness in order to reduce toxicity in the platform. In this research, we analyzed data from “r/VRChat,” to understand how users perceive the system. We found that users interpret the system differently. Problems in its implementation cause distrust. The trust ranks, while intended to promote positive interactions, can actually lead to stereotyping and discourage communication between users of different ranks. The hierarchical structure within the ranks exacerbates discrimination and conflicts, particularly against the low-ranked users. We further discuss that trust ranks present challenges to newcomers and contribute to a competitive atmosphere that hinders the formation of less toxic norms. Finally, we provide implications for the future design of similar systems.

Researchers increasingly explore deploying brain-computer interfaces (BCIs) for able-bodied users, with the motivation of accessing mental states more directly than allowed by existing body-mediated interaction.This motivation seems to contradict the long-standing HCI emphasis on embodiment, namely the general claim that the body is crucial for cognition.This paper addresses this apparent contradiction through a review of insights from embodied cognition and interaction.We first critically examine the recent interest in BCIs and identify the extent cognition in the brain is integrated with the wider body as a central concern for research.We then define the implications of an integrated view of cognition for interface design and evaluation.A counterintuitive conclusion we draw is that embodiment per se should not imply a preference for body-mediated interaction over BCIs.It can instead guide research by 1) providing bodygrounded explanations for BCI performance, 2) proposing evaluation considerations that are neglected in modular views of cognition, and 3) through the direct transfer of its design insights to BCIs.We finally reflect on HCI's understanding of embodiment and identify the neural dimension of embodiment as hitherto overlooked.

A novel participatory livestreaming genre, Avatar Collective Streaming, is gaining traction. Beyond traditional audience participation mechanisms, such as chat messages or commands, these livestreams allow viewers to join, socially interact, and participate through avatars in a software-driven virtual scenario without requiring the presence of a human streamer. From observations of livestreaming sessions and interviews with regular users (n=23), we define the main features of this novel form of livestreaming and we analyze what makes it engaging for participants, as well as factors that limit the user experience. The use of avatars affords a strong sense of presence and participation, and together with the absence of a streamer, it shapes new forms of sociability, being together, and belonging. Challenges include repetitive content and overcrowding. We provide design implications for synchronous collective participation scenarios and opportunities for software-driven content generation.

Phantom tactile sensation (PTS) is usually experienced by participants in laboratory settings with the assistance and supervision of professionals. Extensive reports from users demonstrate they experience PTS on commercially available virtual reality (VR) platforms. We gathered and analyzed 2885 posts by 1408 users to understand how users obtain PTS and how they evaluate their PTS experience. We observed that users experience PTS in three ways: 1) starting to feel it naturally, 2) intentionally developing the ability to experience PTS, and 3) feeling it under substance use. Users perceive the sensation differently. Many people perceive PTS as positive, enhancing their immersion and bringing people closer. While other users consider it a negative experience as it exacerbates harassment issues, or the sensation itself is negative, even painful. We discuss the perceived causes of PTS and how social VR conditions users' evaluation of their PTS experience. We further reflect on PTS from the perspective of the risk of VR use in real-life scenarios. Finally, we provide design implications on employing PTS to enhance users' VR experience and reduce the negatives PTS brings.

Our digital life consists of activities that are organized around tasks and exhibit different user states in the digital contexts around these activities. Previous works have shown that digital activity monitoring can be used to predict entities that users will need to perform digital tasks. There have been methods developed to automatically detect the tasks of a user. However, these studies typically support only specific applications and tasks, and relatively little research has been conducted on real-life digital activities. This article introduces user state modeling and prediction with contextual information captured as entities, recorded from real-world digital user behavior, called entity footprinting —a system that records users’ digital activities on their screens and proactively provides useful entities across application boundaries without requiring explicit query formulation. Our methodology is to detect contextual user states using latent representations of entities occurring in digital activities. Using topic models and recurrent neural networks, the model learns the latent representation of concurrent entities and their sequential relationships. We report a field study in which the digital activities of 13 people were recorded continuously for 14 days. The model learned from this data is used to (1) predict contextual user states and (2) predict relevant entities for the detected states. The results show improved user state detection accuracy and entity prediction performance compared to static, heuristic, and basic topic models. Our findings have implications for the design of proactive recommendation systems that can implicitly infer users’ contextual state by monitoring users’ digital activities and proactively recommending the right information at the right time.

Individuals are known to differ in cognitive abilities, affecting their behavior and information processing in digital environments. However, we have a limited understanding of which behaviors are affected, how, and whether some features extracted from digital behavior can predict cognitive abilities. Consequently, researchers may miss opportunities to design and support individuals with personalized experiences and detect those who may benefit from additional interventions. To characterize digital behaviors, we collected 24/7 screen recordings, input behavior, and operating system data from the laptops of 20 adults for two weeks. We use cognitive test results from the same individuals to characterize their cognitive abilities: psychomotor speed, processing speed, selective attention, working memory, and fluid intelligence. Our results from regression analysis, path modeling, and machine learning experiments show that cognitive abilities are associated with differences in digital behavior and that naturalistic behavioral data can predict the cognitive abilities of individuals with small error rates. Our findings suggest naturalistic interaction data as a novel source for modeling cognitive differences.

Interpersonal touch is critical for social-emotional development and presents a powerful modality for communicating emotions. Virtual agents of the future could capitalize on touch to establish social bonds with humans and facilitate cooperation in virtual reality (VR). We studied whether the emotional expression of a virtual agent would affect the way humans touch the agent. Participants were asked to hold a pressure-sensing tube presented as the agent’s arm in VR. Upon seeing the agent’s emotional expression change, participants briefly squeezed the arm. The effect of emotional expressions on affective state was measured using self-reported valence and arousal as well as physiology-based indices. Onset, duration, and intensity of the squeeze were recorded to examine the haptic responses. Emotional expression of agents affected squeeze intensity and duration through changes in emotional perception and experience. Haptic responses may thus provide an implicit measure of persons’ experience towards their virtual companion.

Gestational diabetes mellitus (GDM) has considerable and increasing health effects as it raises both the mother’s and offspring’s risk for short- and long-term health problems. GDM can usually be treated with a healthier lifestyle, such as appropriate dietary modifications and engaging insufficient physical activity. While telemedicine interventions requiring weekly or more frequent feedback from health care professionals have shown the potential to improve glycemic control amongst women with GDM, apps without extensive input from health care professionals are limited and have not shown to be effective. We aimed to improve the efficacy of GDM self-management apps by exploring desirable features in a review. We derived six desirable features from the multidisciplinary literature and we evaluated the state of implementation of these features in existing GDM apps. The results showed that features for increasing competence to manage GDM and for providing social support were largely lacking.

With the advent of consumer grade virtual reality (VR) headsets and physiological measurement devices, new possibilities for mediated social interaction emerge enabling the immersion to environments where the visual features react to the users’ physiological activation. In this study, we investigated whether and how individual and interpersonally shared biofeedback (visualised respiration rate and frontal asymmetry of electroencephalography, EEG) enhance synchrony between the users’ physiological activity and perceived empathy towards the other during a compassion meditation exercise carried out in a social VR setting. The study was conducted as a laboratory experiment$(\text{N} = 72)$(N=72)employing a Unity3D-based Dynecom immersive social meditation environment and two amplifiers to collect the psychophysiological signals for the biofeedback. The biofeedback on empathy-related EEG frontal asymmetry evoked higher self-reported empathy towards the other user than the biofeedback on respiratory activation, but the perceived empathy was highest when both feedbacks were simultaneously presented. In addition, the participants reported more empathy when there was stronger EEG frontal asymmetry synchronization between the users. The presented results inform the field of affective computing on the possibilities that VR offers for different applications of empathic technologies.

We study the effect of contextual information obtained from a user’s digital trace on Web search performance. Contextual information is modeled using Dirichlet–Hawkes processes (DHP) and used in augmenting Web search queries. The context is captured by monitoring all naturally occurring user behavior using continuous 24/7 recordings of the screen and associating the context with the queries issued by the users. We report a field study in which 13 participants installed a screen recording and digital activity monitoring system on their laptops for 14 days, resulting in data on all Web search queries and the associated context data. A query augmentation (QAug) model was built to expand the original query with semantically related terms. The effects of context window and source were determined by training context models with temporally varying context windows and varying application sources. The context models were then utilized to re-rank the QAug model. We evaluate the context models by using the Web document rankings of the original query as a control condition compared against various experimental conditions: (1) a search context condition in which the context was sourced from search history; (2) a non-search context condition in which the context was sourced from all interactions excluding search history; (3) a comprehensive context condition in which the context was sourced from both search and non-search histories; and (4) an application-specific condition in which the context was sourced from interaction histories captured on a specific application type. Our results indicated that incorporating more contextual information significantly improved Web search rankings as measured by the positions of the documents on which users clicked in the search result pages. The effects and importance of different context windows and application sources, along with different query types are analyzed, and their impact on Web search performance is discussed.

End-user-programmable smart-home toolkits have engendered excitement in recent years. However, modern homes already cater quite well to users’ needs, and genuinely new needs for smart-home automation seldom arise. Acknowledging this challenging starting point, we conducted a six-week in-the-wild study of smart-home toolkits with four carefully recruited technology-savvy families. Interleaved with free toolkit use in the home were several creativity workshops to facilitate ideation and programming. We evaluated use experiences at the end of the six weeks. Even with extensive facilitation, families faced difficulties in identifying needs for smart-home automation, except for social needs that emerged in all the families. We present analysis of those needs and discuss how end-user-programmable toolkits could better engage with both those household members who design new automated functions and those who merely ‘use’ them.

Everyday digital tasks can highly benefit from systems that recommend the right information to use at the right time. However, existing solutions typically support only specific applications and tasks. In this demo, we showcase EntityBot, a system that captures context across application boundaries and recommends information entities related to the current task. The user’s digital activity is continuously monitored by capturing all content on the computer screen using optical character recognition. This includes all applications and services being used and specific to individuals’ computer usages such as instant messaging, emailing, web browsing, and word processing. A linear model is then applied to detect the user’s task context to retrieve entities such as applications, documents, contact information, and several keywords determining the task. The system has been evaluated with real-world tasks, demonstrating that the recommendation had an impact on the tasks and led to high user satisfaction.

Exploratory search starts with ill-defined goals and involves browsing, learning, and formulating new targets for search. To fluidly support such dynamic search behaviours, we focus on devising interactive visual facets (IVF), visualising information facets to support user comprehension and control of the information space. To do this, we reviewed existing faceted search interfaces and derived two design requirements (DR) that have not been fully addressed to support fluid interactions in exploratory search. We then exemplified the requirements through devising an IVF tool, which coordinates a linear and a categorical facet representing the distribution and summarisation of items, respectively, and providing context for faceted exploration (DR1). To support rapid transitions between search criteria (DR2), the tool introduces a novel design concept of using facets to select items without filtering the item space. Particularly, we propose a filter-swipe technique that enables users to drag a categorical facet value sequentially over linear facet bars to view the items in the intersection of the two facets along with the categorical facet dynamically summarizing the items in the interaction. A user study of 11 participants with realistic email search tasks shows that dynamic suggestions through the timeline navigation can help discover useful suggestions for search; the novel design concept was favoured over using facet values as filters. Based on these practices, we derive IVF design implications for fluid, exploratory searches.

Digital tools for collocated synchronous communication, like backchannels and question management systems, are increasingly used in events and education to support interaction between performer(s) and audience. Discussions in the communication tool are prone to various unexpected interruptions which violate against the communication tools' original purpose, i.e. violating performers' expected norms. We study such norm violations and related performers' coping approaches in two 15-week-long field experiments in two elementary schools. Our results demonstrate how the collocated synchronous tools form dual-channel communication environment where both physical and digital channel are in use at the same time. We show how norm violations 'spill over' from their onset in one channel to continue in the other. However, we observed performers coping approaches were primarily focused in the physical channel. This resulted in an asymmetry of presence and delayed the performers' ability to act on violations. Our findings show that the channels in dual-channel communication are tightly interwoven and must be considered as an integrated communication space instead of two co-occurring but separate channels. We provide a framework to study norm violations and discuss implications of this framework on the design of these systems.

Recommender systems can support everyday digital tasks by retrieving and recommending useful information contextually. This is becoming increasingly relevant in services and operating systems. Previous research often focuses on specific recommendation tasks with data captured from interactions with an individual application. The quality of recommendations is also often evaluated addressing only computational measures of accuracy, without investigating the usefulness of recommendations in realistic tasks. The aim of this work is to synthesize the research in this area through a novel approach by (1) demonstrating comprehensive digital activity monitoring, (2) introducing entity-based computing and interaction, and (3) investigating the previously overlooked usefulness of entity recommendations and their actual impact on user behavior in real tasks. The methodology exploits context from screen frames recorded every 2 seconds to recommend information entities related to the current task. We embodied this methodology in an interactive system and investigated the relevance and influence of the recommended entities in a study with participants resuming their real-world tasks after a 14-day monitoring phase. Results show that the recommendations allowed participants to find more relevant entities than in a control without the system. In addition, the recommended entities were also used in the actual tasks. In the discussion, we reflect on a research agenda for entity recommendation in context, revisiting comprehensive monitoring to include the physical world, considering entities as actionable recommendations, capturing drifting intent and routines, and considering explainability and transparency of recommendations, ethics, and ownership of data.

Web searches often originate from conversations in which people engage before they perform a search. Therefore, conversations can be a valuable source of context with which to support the search process. We investigate whether spoken input from conversations can be used as a context to improve query auto-completion. We model the temporal dynamics of the spoken conversational context preceding queries and use these models to re-rank the query auto-completion suggestions. Data were collected from a controlled experiment and comprised conversations among 12 participant pairs conversing about movies or traveling. Search query logs during the conversations were recorded and temporally associated with the conversations. We compared the effects of spoken conversational input in four conditions: a control condition without contextualization; an experimental condition with the model using search query logs; an experimental condition with the model using spoken conversational input; and an experimental condition with the model using both search query logs and spoken conversational input. We show the advantage of combining the spoken conversational context with the Web-search context for improved retrieval performance. Our results suggest that spoken conversations provide a rich context for supporting information searches beyond current user-modeling approaches.

Understanding the quality of insight has become increasingly important with the trend of allowing users to post comments during visual exploration, yet approaches for qualifying insight are rare. This article presents a case study to investigate the possibility of characterizing the quality of insight via the interactions performed. To do this, we devised the interaction of a visualization tool-MediSyn-for insight generation. MediSyn supports five types of interactions: selecting, connecting, elaborating, exploring, and sharing. We evaluated MediSyn with 14 participants by allowing them to freely explore the data and generate insights. We then extracted seven interaction patterns from their interaction logs and correlated the patterns to four aspects of insight quality. The results show the possibility of qualifying insights via interactions. Among other findings, exploration actions can lead to unexpected insights; the drill-down pattern tends to increase the domain values of insights. A qualitative analysis shows that using domain knowledge to guide exploration can positively affect the domain value of derived insights. We discuss the study's implications, lessons learned, and future research opportunities.

Abstract The outcome of exploratory information retrieval is not only dependent on the effectiveness of individual responses to a set of queries, but also on relevant information retrieved during the entire exploratory search session. We study the effect of search assistance, operationalized as an interactive faceted query suggestion, for both whole‐session effectiveness and engagement through interactive faceted query suggestion. A user experiment is reported, where users performed exploratory search tasks, comparing interactive faceted query suggestion and a control condition with only conventional typed‐query interaction. Data comprised of interaction and search logs show that the availability of interactive faceted query suggestion substantially improves whole‐session effectiveness by increasing recall without sacrificing precision. The increased engagement with interactive faceted query suggestion is targeted to direct situated navigation around the initial query scope, but is not found to improve individual queries on average. The results imply that research in exploratory search should focus on measuring and designing tools that engage users with directed situated navigation support for improving whole‐session performance.

The term implicit interaction is often used to denote interactions that differ from traditional purposeful and attention demanding ways of interacting with computers. However, there is a lack of agreement about the term's precise meaning. This paper develops implicit interaction further as an analytic concept and identifies the methodological challenges related to HCI's particular design orientation. We first review meanings of implicit as unintentional, attentional background, unawareness, unconsciousness and implicature, and compare them in regards to the entity they qualify, the design motivation they emphasize and their constructive validity for what makes good interaction. We then demonstrate how the methodological challenges can be addressed with greater precision by using an updated, intentionality-based definition that specifies an input-effect relationship as the entity of implicit. We conclude by identifying a number of new considerations for design and evaluation, and by reflecting on the concepts of user and system agency in HCI.

Despite advances in the past few decades in studying what kind of queries users input to search engines and how to suggest queries for the users, the fundamental question of what makes human cognition able to estimate goodness of query terms is largely unanswered. For example, a person searching information about "cats'' is able to choose query terms, such as "housecat'', "feline'', or "animal'' and avoid terms like "similar'', "variety'', and "distinguish''. We investigated the association between the specificity of terms occurring in documents and human brain activity measured via electroencephalography (EEG). We analyzed the brain activity data of fifteen participants, recorded in response to reading terms from Wikipedia documents. Term specificity was shown to be associated with the amplitude of evoked brain responses. The results indicate that by being able to determine which terms carry maximal information about, and can best discriminate between, documents, people have the capability to enter good query terms. Moreover, our results suggest that the effective query term selection process, often observed in practical search behavior studies, has a neural basis. We believe our findings constitute an important step in revealing the cognitive processing behind query formulation and evaluating informativeness of language in general.

The use of implicit relevance feedback from neurophysiology could deliver effortless information retrieval. However, both computing neurophysiologic responses and retrieving documents are characterized by uncertainty because of noisy signals and incomplete or inconsistent representations of the data. We present the first-of-its-kind, fully integrated information retrieval system that makes use of online implicit relevance feedback generated from brain activity as measured through electroencephalography (EEG), and eye movements. The findings of the evaluation experiment (N = 16) show that we are able to compute online neurophysiology-based relevance feedback with performance significantly better than chance in complex data domains and realistic search tasks. We contribute by demonstrating how to integrate in interactive intent modeling this inherently noisy implicit relevance feedback combined with scarce explicit feedback. Although experimental measures of task performance did not allow us to demonstrate how the classification outcomes translated into search task performance, the experiment proved that our approach is able to generate relevance feedback from brain signals and eye movements in a realistic scenario, thus providing promising implications for future work in neuroadaptive information retrieval (IR).

Understanding users' search behavior has largely relied on the information available from search engine logs, which provide limited information about the contextual factors affecting users' behavior. Consequently, questions such as how users' intentions, task goals, and substances of the users' tasks affect search behavior, as well as what triggers information needs, remain largely unanswered. We report an experiment in which naturalistic information search behavior was captured by analyzing 24/7 continuous recordings of information on participants' computer screens. Written task diaries describing the participants' tasks were collected and used as real‐life task contexts for further categorization. All search tasks were extracted and classified under various task categories according to users' intentions, task goals, and substances of the tasks. We investigated the effect of different task categories on three behavioral factors: search efforts, content‐triggers, and application context. Our results suggest four findings: (i) Search activity is integrally associated with the users' creative processes. The content users have seen prior to searching more often triggers search, and is used as a query, within creative tasks. (ii) Searching within intellectual and creative tasks is more time‐intensive, while search activity occurring as a part of daily routine tasks is associated with more frequent searching within a search task. (iii) Searching is more often induced from utility applications in tasks demanding a degree of intellectual effort. (iv) Users' leisure information‐seeking activity is occurring inherently within social media services or comes from social communication platforms. The implications of our findings for information access and management systems are discussed.

Conversations among people involve solving disputes, building common ground, and reinforce mutual beliefs and assumptions. Conversations often require external information that can support these human activities. In this paper, we study how a spoken conversation can be supported by a proactive search agent that listens to the conversation, detects entities mentioned in the conversation, and proactively retrieves and presents information related to the conversation. A total of 24 participants (12 pairs) were involved in informal conversations, using either the proactive search agent or a control condition that did not support conversational analysis or proactive information retrieval. Data comprising transcripts, interaction logs, questionnaires, and interviews indicated that the proactive search agent effectively augmented the conversations, affected the conversations' topical structure, and reduced the need for explicit search activity. The findings also revealed key challenges in the design of proactive search systems that assist people in natural conversations.

While ordinary touchscreen-based interfaces on urban explorer applications draw much of a user's attention onto the screen, visual and audio-haptic augmented reality interfaces have emerged as the two main streams for enabling direct focus on the surroundings. However, neither interface alone satisfies users in the highly dynamic urban environment. This research investigates how the two complementary augmentation can coexist on one system and how people adapt to the situation by selecting the more suitable interface. A prototype was deployed in a field experiment in which participants explored points of interest in an urban environment with both interfaces. The engagement with the surroundings was compared with a touchscreen-based application. Most participants spontaneously switched between the two interfaces, which manifests the value of the availability of both interfaces on one system. The results point at the situated advantages of either interface and reveal the users' preferences when both interfaces are available.

We introduce the concept of hypercue, a complement to the hyperlink in the form of an interactive representation of real-world entities, e.g. persons, places, concepts, providing personalized access points to information. As a pendant to the hyperlink, hypercues create opportunities to flexibly discover, store and share information, organize one's thoughts and gain insights of the data. We explore the design space of interaction techniques supporting entity-based information exploration by reviewing recent examples of such work. We reflect on these through the lens of eight essential features of exploratory search systems, to devise generalizable design principles. Our main contribution is a design template describing the hypercue. It consists of a minimal set of affordances that ensure all important features for supporting exploratory search can be addressed, while leaving enough design space to facilitate integration within a variety of systems. We finally describe the rationale behind the design template and discuss its implications.

Industrial applications can benefit considerably from the overwhelming amount of still growing resources such as websites, images, texts, and videos that the internet offers today. The resulting Big Data Problem does not only consist of handling this immense volume of data. Moreover, data needs to be processed, cleaned, and presented in a user-friendly, intuitive, and interactive way. This workshop addresses visualization and user interaction challenges posed by the four V's: Volume (huge data amounts in the range of tera and petabytes), Velocity (the speed in which data is created, processed, and analysed), Variety (the different heterogeneous data types, sources, and formats), and Veracity (authenticity and validity of data). Big Data driven interfaces combine suitable backend and frontend technologies as well as automatic and semi-automatic approaches in order to analyze data in various business contexts. An important aspect is human intervention in developing and training data-driven applications (human in the loop). Our focus is on Visual Big Data Interfaces in industrial contexts such as e-commerce, e-learning and business intelligence. We address interfaces for three important user groups: data scientists, data workers and end users.

Previous work has shown that large high resolution displays (LHRDs) can enhance collaboration between users. As LHRDs allow free movement in front of the screen, an understanding of movement behavior is required to build successful interfaces for these devices. This paper presents Pac-Many; a multiplayer version of the classical computer game Pac-Man to study group dynamics when using LHRDs. We utilized smartphones as game controllers to enable free movement while playing the game. In a lab study, using a 4m × 1m LHRD, 24 participants (12 pairs) played Pac-Many in collaborative and competitive conditions. The results show that players in the collaborative condition divided screen space evenly. In contrast, competing players stood closer together to avoid benefits for the other player. We discuss how the nature of the task is important when designing and analyzing collaborative interfaces for LHRDs. Our work shows how to account for the spatial aspects of interaction with LHRDs to build immersive experiences.

The effects of bio-adaptation-based visual cues in conveying affective information was studied while conducting social meditation in a virtual reality (VR) environment. In a laboratory study, 22 dyads practiced a short duration, modified version of empathy-evoking compassion meditation. The immersive VR provided electroencephalography (EEG) -based real-time visual feedback, as changing color, on the user's brain activation; the breathing rate of the users was visualized as a movement cue. In addition, the synchronization of the feedback signals between the users was also visualized. Initial results suggest that the perceived affective interdependence, a component of social presence, was increased when both of the bio-adaptive visual feedbacks were presented to the users, compared to when there was no visual feedback. Also, the EEG-based visual feedback led to higher affective interdependence than the respiration based visual feedback. The findings suggest that affective contagion may occur by mediated adaptive cues that are based on physiological signals.

This workshop will discuss how different multimedia applications and interaction techniques have been employed to support public sociality---from collocated interactions to social awareness and feeling of community. Participants will discuss recent work and join hands-on design activities to envision new ways for potential users to discover and learn how to use ubiquitous technologies for opportunistic social interactions.

Motivation: Precision medicine requires the ability to predict the efficacies of different treatments for a given individual using high-dimensional genomic measurements. However, identifying predictive features remains a challenge when the sample size is small. Incorporating expert knowledge offers a promising approach to improve predictions, but collecting such knowledge is laborious if the number of candidate features is very large. Results: We introduce a probabilistic framework to incorporate expert feedback about the impact of genomic measurements on the outcome of interest and present a novel approach to collect the feedback efficiently, based on Bayesian experimental design. The new approach outperformed other recent alternatives in two medical applications: prediction of metabolic traits and prediction of sensitivity of cancer cells to different drugs, both using genomic features as predictors. Furthermore, the intelligent approach to collect feedback reduced the workload of the expert to approximately 11%, compared to a baseline approach. Availability and implementation: Source code implementing the introduced computational methods is freely available at https://github.com/AaltoPML/knowledge-elicitation-for-precision-medicine. Supplementary information: Supplementary data are available at Bioinformatics online.

Collaborative and co-located information access is becoming increasingly common. However, fairly little attention has been devoted to the design of ubiquitous computing approaches for spontaneous exploration of large information spaces enabling co-located collaboration. We investigate whether an entity-based user interface provides a solution to support co-located search on heterogeneous devices. We present the design and implementation of QueryTogether, a multi-device collaborative search tool through which entities such as people, documents, and keywords can be used to compose queries that can be shared to a public screen or specific users with easy touch enabled interaction. We conducted mixed-methods user experiments with twenty seven participants (nine groups of three people), to compare the collaborative search with QueryTogether to a baseline adopting established search and collaboration interfaces. Results show that QueryTogether led to more balanced contribution and search engagement. While the overall s-recall in search was similar, in the QueryTogether condition participants found most of the relevant results earlier in the tasks, and for more than half of the queries avoided text entry by manipulating recommended entities. The video analysis demonstrated a more consistent common ground through increased attention to the common screen, and more transitions between collaboration styles. Therefore, this provided a better fit for the spontaneity of ubiquitous scenarios. QueryTogether and the corresponding study demonstrate the importance of entity based interfaces to improve collaboration by facilitating balanced participation, flexibility of collaboration styles and social processing of search entities across conversation and devices. The findings promote a vision of collaborative search support in spontaneous and ubiquitous multi-device settings, and better linking of conversation objects to searchable entities.

We explore electroencephalography (EEG), electrodermal activity (EDA), and electrocardiography (ECG) as valid sources to infer humor appraisal in a realistic environment. We report on an experiment in which 25 participants browsed a popular user-generated humorous content website while their physiological responses were recorded. We build predictive models to infer the participants’ appraisal of the humorousness of the content and demonstrate that the fusion of several physiological signals can lead to classification performances up to 0.73 in terms of the area under the ROC curve (AUC). We identify that the most discriminative changes in physiological signals happen at the later stages of the information consumption process, reflected in changes on the upper EEG frequency bands, higher levels of EDA, and heart-rate acceleration. Additionally, we present a comprehensive analysis by benchmarking the predictive power of each of the physiological signals separately, and by comparing them to state-of-the-art facial recognition algorithms based on facial video recordings. The classification performance ranges from 0.88 (in terms of AUC) when combining physiological signals and video recordings, to 0.55 when using ECG signals alone.

Exploratory search requires the system to assist the user in comprehending the information space and expressing evolving search intents for iterative exploration and retrieval of information. We introduce interactive intent modeling, a technique that models a user’s evolving search intents and visualizes them as keywords for interaction. The user can provide feedback on the keywords, from which the system learns and visualizes an improved intent estimate and retrieves information. We report experiments comparing variants of a system implementing interactive intent modeling to a control system. Data comprising search logs, interaction logs, essay answers, and questionnaires indicate significant improvements in task performance, information retrieval performance over the session, information comprehension performance, and user experience. The improvements in retrieval effectiveness can be attributed to the intent modeling and the effect on users’ task performance, breadth of information comprehension, and user experience are shown to be dependent on a richer visualization. Our results demonstrate the utility of combining interactive modeling of search intentions with interactive visualization of the models that can benefit both directing the exploratory search process and making sense of the information space. Our findings can help design personalized systems that support exploratory information seeking and discovery of novel information.

We explore the combination of above-surface sensing with eye tracking to facilitate concurrent interaction with multiple regions on touch screens. Conventional touch input relies on positional accuracy, thereby requiring tight visual monitoring of one's own motor action. In contrast, above-surface sensing and eye tracking provides information about how user's hands and gaze are distributed across the interface. In these situations we facilitate interaction by 1) showing the visual feedback of the hand hover near user's gaze point and 2) decrease the requisite of positional accuracy by employing gestural information. We contribute input and visual feedback techniques that combine these modalities and demonstrate their use in example applications. A controlled study showed the effectiveness of our techniques for manipulation tasks against conventional touch, while the effectiveness in acquisition tasks depended on the amount of mid-air motion, leading to our conclusion that the techniques can benefit interacting with multiple interface regions.

Online crowds can help infuse creativity into the design process, but traditional strategies for leveraging them, such as large-scale ideation platforms, require time and organizational effort in order to obtain results. We propose a new method for crowd-based ideation that simplifies the process by having smaller crowds join in-person ideators during synchronous creative sessions. Our system Crowdboard allows online crowds to provide real-time creative input during early-stage design activities, such as brainstorming or concept mapping. The system enables in-person ideators to develop ideas on a physical or digital whiteboard which is augmented with real-time creative input from online participants who see and hear a live broadcast of the meeting. We validate Crowdboard via two user studies in which dyads of in-person ideators brainstormed with the help of crowd ideators. Our studies suggest that Crowdboard can effectively enhance ongoing brainstorming sessions, but also revealed key challenges for how to better facilitate interactions among in-person and crowd ideators.

Computer-mediated communication (CMC) tools are used to increase social interaction in collocated settings. Recent research has been primarily constructive (oriented to building of systems) or phenomenon-driven (serving attempts to understand interactions in collocated CMC). The paper contributes a theory-driven approach and examines collocated CMC as a Habermasean "public sphere": a space that supports inclusive, civil, and rational discussion. An in-the-wild experimental study comparing CMC with face-to-face (F2F) communication enabled ascertaining that CMC is more inclusive than F2F communication. Respectfulness levels did not differ but were established differently: via collective construction of a common narrative in F2F and through quick reactions in CMC. Similarly, while rationality figures were on a par, F2F communication allowed participants to justify their claims better. The article discusses how a theory-based approach can strengthen phenomenon-driven research with new conceptual frames and measurement tools, and steer constructive research with a normative framework.

There is a blind spot in HCI's evaluation methodology: we rarely consider the implications of the fact that a prototype can never be fully evaluated in a study. A prototype under study exists firmly in the present world, in the circumstances created in the study, but its real context of use is a partially unknown future state of affairs. This present-future gap is implicit in any evaluation of prototypes, be they usability tests, controlled experiments, or field trials. A carelessly designed evaluation may inadvertently evaluate the wrong futures, contexts, or user groups, thereby leading to false conclusions and expensive design failures. The essay analyses evaluation methodology from this perspective, illuminating how to mitigate the present-future gap.

We present visual re-ranking, an interactive visualization technique for multi-aspect information retrieval. In multi-aspect search, the information need of the user consists of more than one aspect or query simultaneously. While visualization and interactive search user interface techniques for improving user interpretation of search results have been proposed, the current research lacks understanding on how useful these are for the user: whether they lead to quantifiable benefits in perceiving the result space and allow faster, and more precise retrieval. Our technique visualizes relevance and document density on a two-dimensional map with respect to the query phrases. Pointing to a location on the map specifies a weight distribution of the relevance to each of the query phrases, according to which search results are re-ranked. User experiments compared our technique to a uni-dimensional search interface with typed query and ranked result list, in perception and retrieval tasks. Visual re-ranking yielded improved accuracy in perception, higher precision in retrieval and overall faster task execution. Our findings demonstrate the utility of visual re-ranking, and can help designing search user interfaces that support multi-aspect search.

Providing accurate predictions is challenging for machine learning algorithms when the number of features is larger than the number of samples in the data. Prior knowledge can improve machine learning models by indicating relevant variables and parameter values. Yet, this prior knowledge is often tacit and only available from domain experts. We present a novel approach that uses interactive visualization to elicit the tacit prior knowledge and uses it to improve the accuracy of prediction models. The main component of our approach is a user model that models the domain expert's knowledge of the relevance of different features for a prediction task. In particular, based on the expert's earlier input, the user model guides the selection of the features on which to elicit user's knowledge next. The results of a controlled user study show that the user model significantly improves prior knowledge elicitation and prediction accuracy, when predicting the relative citation counts of scientific documents in a specific domain.

We demonstrate proactive information retrieval via screen surveillance. A user's digital activities are continuously monitored by capturing all content on a user's screen using optical character recognition. This includes all applications and services being exploited and relies on each individual user's computer usage, such as their Web browsing, emails, instant messaging, and word processing. Topic modeling is then applied to detect the user's topical activity context to retrieve information. We demonstrate a system that proactively retrieves information from a user's activity history being observed on the screen when the user is performing unseen activities on a personal computer. We report an evaluation with ten participants that shows high user satisfaction and retrieval effectiveness. Our demonstration and experimental results show that surveillance of a user's screen can be used to build an extremely rich model of a user's digital activities across application boundaries and enable effective proactive information retrieval.

This paper describes the development and evaluation of Undulating Covers (UnCovers), mobile interfaces that can change their surface texture to transmit information. The Pin Array UnCover incorporates sinusoidal ridges controlled by servomotors, which can change their amplitude and granularity. The Mylar UnCover is a more organic interface that exploits the buckling properties of Mylar, using muscle wires, to change the texture granularity. The prototype development process and evaluations show that very low frequency texture changes, using amplitude or granularity, can be distinguished with high levels of accuracy. Since small changes are perceptible, it is possible to incorporate such interfaces into mobile devices without drastically increasing their size or actuation requirements. Finally, ratings from participants indicate that UnCovers would be appropriate for attention-grabbing, or caring and supportive interpersonal messages.

Following recent advances in visual interfaces for search, we investigate how to visualize activity traces to support collaborative information-seeking. We implemented a prototype system that visualizes traces of three types of activities (queries typed, articles bookmarked, and interested keywords) on top of a recent visual search system. The current interface of the system provides an interactive keyword visualization to support exploratory search. We designed two icons to visualize user interactions with the keywords and articles. We also implemented a time line to explicitly display the issued queries and additional details about the interactions. We conducted a longitudinal user study to evaluate the usability of these visualizations. We found that the visualization of traces of issued queries and bookmarked articles help the users to better monitor the progress of the collaborators, increase collaboration, and reassure their findings. Traces of interested keywords support sub-topic identification. Furthermore, activity traces of collaborators help the users to learn about the system features. The findings and the novel collaborative system provide valuable insights and tools into future research on designing interfaces for collaborative information-seeking.

We present a preliminary design study for utilizing eye tracking to support interacting with a multi-document visualization. Complex information seeking tasks can involve collection and comparison of multiple documents, resulting in long and sustained search sessions. The sustained and evolving nature of the session also provides the search interface the opportunity to gather information on the user state and interaction history, which can be used to adapt the information content and representation. We designed a system to evaluate how eye tracking information can be used for adapting the visual salience of information entities. The interface features documents and related keywords that are arranged in a radial layout configuration called the intent radar. Reading history and visual attention, as registered by eye tracking data, are respectively used to trace read items and for visual cueing. We evaluated the interface with 16 participants to gather subjective feedback about specific components and features of the interface. The overall results show that the interface and gaze-related appearance of keywords was positively received by the users.

BACKGROUND: Dispersed biomedical databases limit user exploration to generate structured knowledge. Linked Data unifies data structures and makes the dispersed data easy to search across resources, but it lacks supporting human cognition to achieve insights. In addition, potential errors in the data are difficult to detect in their free formats. Devising a visualization that synthesizes multiple sources in such a way that links between data sources are transparent, and uncertainties, such as data conflicts, are salient is challenging. RESULTS: To investigate the requirements and challenges of uncertainty-aware visualizations of linked data, we developed MediSyn, a system that synthesizes medical datasets to support drug treatment selection. It uses a matrix-based layout to visually link drugs, targets (e.g., mutations), and tumor types. Data uncertainties are salient in MediSyn; for example, (i) missing data are exposed in the matrix view of drug-target relations; (ii) inconsistencies between datasets are shown via overlaid layers; and (iii) data credibility is conveyed through links to data provenance. CONCLUSIONS: Through the synthesis of two manually curated datasets, cancer treatment biomarkers and drug-target bioactivities, a use case shows how MediSyn effectively supports the discovery of drug-repurposing opportunities. A study with six domain experts indicated that MediSyn benefited the drug selection and data inconsistency discovery. Though linked publication sources supported user exploration for further information, the causes of inconsistencies were not easy to find. Additionally, MediSyn could embrace more patient data to increase its informativeness. We derive design implications from the findings.

Previous research investigated how to leverage the new type of social data available on the web, e.g., tags, ratings and reviews, in recommending and personalizing information. However, previous works mainly focused on predicting ratings using collaborative filtering or quantifying personalized ranking quality in simulations. As a consequence, the effect of social information in user's information search and information-selection behavior remains elusive. The objective of our research is to investigate the effects of social information on users’ interactive search and information-selection behavior. We present a computational method and a system implementation combining different graph overlays: social, personal and search-time user input that are visualized for the user to support interactive information search. We report on a controlled laboratory experiment, in which 24 users performed search tasks using three system variants with different graphs as overlays composed from the largest publicly available social content and review data from Yelp: personal preferences, tags combined with personal preferences, and tags and social ratings combined with personal preferences. Data comprising search logs, questionnaires, simulations, and eye-tracking recordings show that: 1) the search effectiveness is improved by using and visualizing the social rating information and the personal preference information as compared to content-based ranking. 2) The need to consult external information before selecting information is reduced by the presentation of the effects of different overlays on the search results. Search effectiveness improvements can be attributed to the use of social rating and personal preference overlays, which was also confirmed in a follow-up simulation study. With the proposed method we demonstrate that social information can be incorporated to the interactive search process by overlaying graphs representing different information sources. We show that the combination of social rating information and personal preference information improves search effectiveness and reduce the need to consult external information. Our method and findings can inform the design of interactive search systems that leverage the information available on the social web.

Tools for supporting performer-audience interaction have been gaining increasing interest in HCI community recently. They encompass a wide range of systems from simple polls to live tweeting and backchannel chats. However, a lack of unifying conceptual framework hampers their efficient development and deployment in events. In this paper, we develop a notion of live participation and present a live participation system that aims to capture performer-audience interaction systems' salient design-relevant characteristics. With user studies, we identify central characteristics of live participation (RQ1) and explore the diversity of types of live participation situations (RQ2). The identified concepts-extended performance, integration work, and episodes-provide the groundwork for further design of live participation systems and more engaging audience interactions.

Technological support for augmenting the relationship that people establish with remote places has been studied fairly little as the primary focus in telepresence studies is the connection between people. This paper addresses the design challenge for supporting "active place presence" at home. A prototype, Hole in Space, was created to explore the design challenge. A longitudinal study of how an urban couple appropriated the prototype was conducted over the duration of seven months. The paper elaborates on how the Web-mediated design influenced place presence and outlines several aspects that need to be considered when designing for active place presence at home.

Smart glasses suffer from obtrusive or cumbersome interaction techniques. Studies show that people are not willing to publicly use, for example, voice control or mid-air gestures in front of the face. Some techniques also hamper the high degree of freedom of the glasses. In this paper, we derive design principles for socially acceptable, yet versatile, interaction techniques for smart glasses based on a survey of related work. We propose an exemplary design, based on a haptic glove integrated with smart glasses, as an embodiment of the design principles. The design is further refined into three interaction scenarios: text entry, scrolling, and point-and-select. Through a user study conducted in a public space we show that the interaction technique is considered unobtrusive and socially acceptable. Furthermore, the performance of the technique in text entry is comparable to state-of-the-art techniques. We conclude by reflecting on the advantages of the proposed design.

We propose using eye tracking to support interface use with decreased reliance on visual guidance. While the design of most graphical user interfaces take visual guidance during manual input for granted, eye tracking allows distinguishing between the cases when the manual input is conducted with or without guidance. We conceptualize the latter cases as input with uncertainty that require separate handling. We describe the design space of input handling by utilizing input resources available to the system, possible actions the system can realize and various feedback techniques for informing the user. We demonstrate the particular action mechanisms and feedback techniques through three applications we developed for touch interaction on a large screen. We conducted a two stage study of positional accuracy during target acquisition with varying visual guidance, to determine the selection range around a touch point due to positional uncertainty. We also conducted a qualitative evaluation of example applications with participants to identify perceived utility and hand eye coordination challenges while using interfaces with decreased visual guidance.

Virtual reality presents an extraordinary platform for multimodal communication. Haptic technologies have been shown to provide an important contribution to this by facilitating co-presence and allowing affective communication. However, the findings of the affective influences rely on studies that have used myriad different types of haptic technology, making it likely that some forms of tactile feedback are more efficient in communicating emotions than others. To find out whether this is true and which haptic technologies are most effective, we measured user experience during a communication scenario featuring an affective agent and interpersonal touch in virtual reality. Interpersonal touch was simulated using two types of vibrotactile actuators and two types of force feedback mechanisms. Self-reports of subjective experience of the agent’s touch and emotions were obtained. The results revealed that, regardless of the agent’s expression, force feedback actuators were rated as more natural and resulted in greater emotional interdependence and a stronger sense of co-presence than vibrotactile touch.

We present a novel adaptation technique for search engines to better support information-seeking activities that include both lookup and exploratory tasks. Building on previous findings, we describe (1) a classifier that recognizes task type (lookup vs. exploratory) as a user is searching and (2) a reinforcement learning based search engine that adapts accordingly the balance of exploration/exploitation in ranking the documents. This allows supporting both task types surreptitiously without changing the familiar list-based interface. Search results include more diverse results when users are exploring and more precise results for lookup tasks. Users found more useful results in exploratory tasks when compared to a base-line system, which is specifically tuned for lookup tasks.

Meditation in general and mindfulness in particular have been shown to be useful techniques in the treatment of a plethora of ailments, yet they can be challenging for novices. We present RelaWorld: a neuroadaptive virtual reality meditation system that combines virtual reality with neurofeedback to provide a tool that is easy for novices to use yet provides added value even for experienced meditators. Using a head-mounted display, users can levitate in a virtual world by doing meditation exercises. The system measures users' brain activity in real time via EEG and calculates estimates for the level of conCentration and relaxation. These values are then mapped into the virtual reality. In a user study of 43 subjects, we were able to show that the RelaWorld system elicits deeper relaxation, feeling of presence and a deeper level of meditation when compared to a similar setup without head-mounted display or neurofeedback.

Surface computing allows flexible search interaction where users can manipulate the representation of entities recommended for them to create new queries or augment existing queries by taking advantage of increased screen estate and almost physical tactile interaction. We demonstrate a search system based on 1) Direct Manipulation of Entity Representation on Surfaces and 2) Entity Recommendation and Document Retrieval. Entities are modeled as a knowledge-graph and the relevances of entities are computed using the graph structure. Users can manipulate the representation of entities via spatial grouping and assigning preferences on entities. Our contribution can help to design effective information exploration systems that take advantage of large surfaces.

The use of dedicated devices may insufficiently support the variety and subtlety of domestic arrangements: they usually focus on specific aspects (e.g., home automation, health, safety, etc.) and potentially become obsolete, since they are unable to be recomposed and adapted to the needs of new situations. Open-ended and repurposable technologies could better address domestic users' needs. We present T4Tags 2.0, an open-ended toolkit for programming tangible tokens that embed different sensing technologies and can be attached to ordinary objects to create smart behaviors at home. We report findings from a one-day workshop we carried out to explore opportunities of the toolkit.

Augmented Reality (AR) browsers show geo-referenced data in the current view of a user. When the amount of data grows too large, the display quickly becomes cluttered. Clustering items by spatial and semantic attributes can temporarily alleviate the issue, but is not effective against an increasing amount of data. We present an adaptive information density display for AR that balances the amount of presented information against the potential clutter created by placing items on the screen. We use hierarchical clustering to create a level-of-detail structure, in which nodes closer to the root encompass groups of items, while the leaf nodes contain single items. Our method selects items and groups from different levels of this hierarchy based on user-defined preferences and on the amount of visual clutter caused by placing these items. The number of presented items is adapted during user interaction to avoid clutter. We compare our interface to a conventional AR browser interface in a qualitative user study. Users clearly preferred our interface, because it provided a better overview of the data and allowed for easier comparison. In a second study, we evaluated the effect of different degrees of clustering on search and recall tasks. Users generally made fewer errors, when using our interface for a search task, which indicates that the reduced clutter allowed them to stay focused on finding the relevant items.

The term ‘conceptual designing’ refers to an activity that various practitioners already undertake, but for which we lack a clear definition. This article provides that definition and uses an example of a design concept called ‘Manhattan’ to present how exactly this type of process happens. We define conceptual designing as a constructive framing and re-framing activity, which is mediated by and targeted at the creation of new design concepts. Conceptual designing as an approach is valuable for addressing the fuzziness and ambiguity typical of research that explores novel areas with new partners, methods and resources. It is by no means a new phenomenon, and the main contribution of the article is the clarification of conceptual designing as a particular approach to designing and researching. The approach embraces openness, resource-construction and collaboration. We conclude that conceptual designing can be especially useful in research and design projects that bring different kinds of people, organizations, technologies and domains together into the forming of new well-founded proposals for development. The presentation of conceptual designing in this paper is written with an intention to provide designers and researchers with terminology and concepts that they may use to structure their work as well as to become more resourceful in reflecting upon their projects.

Exploratory search is an increasingly important activity yet challenging for users. Although there exists an ample amount of research into understanding exploration, most of the major information retrieval (IR) systems do not provide tailored and adaptive support for such tasks. One reason is the lack of empirical knowledge on how to distinguish exploratory and lookup search behaviors in IR systems. The goal of this article is to investigate how to separate the 2 types of tasks in an IR system using easily measurable behaviors. In this article, we first review characteristics of exploratory search behavior. We then report on a controlled study of 6 search tasks with 3 exploratory—comparison, knowledge acquisition, planning—and 3 lookup tasks—fact‐finding, navigational, question answering. The results are encouraging, showing that IR systems can distinguish the 2 search categories in the course of a search session. The most distinctive indicators that characterize exploratory search behaviors are query length, maximum scroll depth, and task completion time. However, 2 tasks are borderline and exhibit mixed characteristics. We assess the applicability of this finding by reporting on several classification experiments. Our results have valuable implications for designing tailored and adaptive IR systems.

Collaborative idea generation leverages social interactions and knowledge sharing to spark diverse associations and produce creative ideas. Information exploration systems expand the current context by suggesting novel but related concepts. In this paper we introduce InspirationWall, an unobtrusive display that leverages speech recognition and information exploration to enhance an ongoing idea generation session with automatically retrieved concepts that relate to the conversation. We evaluated the system in six idea generation sessions of 20 minutes with small groups of two people. Preliminary results suggest that InspirationWall contrasts the decay of idea productivity over time and can thus represent an effective way to enhance idea generation activities.

Exploratory search confront users with challenges in expressing search intents as the current search interfaces require investigating result listings to identify search directions, iterative typing, and reformulating queries. We present the design of Exploration Wall, a touch-based search user interface that allows incremental exploration and sense-making of large information spaces by combining entity search, flexible use of result entities as query parameters, and spatial configuration of search streams that are visualized for interaction. Entities can be flexibly reused to modify and create new search streams, and manipulated to inspect their relationships with other entities. Data comprising of task-based experiments comparing Exploration Wall with conventional search user interface indicate that Exploration Wall achieves significantly improved recall for exploratory search tasks while preserving precision. Subjective feedback supports our design choices and indicates improved user satisfaction and engagement. Our findings can help to design user interfaces that can effectively support exploratory search on touch devices.

We propose the use of a single continuous gesture as a novel, intuitive, and efficient mechanism to authenticate a secure communication channel. Our approach builds on a novel algorithm for encoding (at least 20-bits) authentication information as a single continuous gesture, referred to as a checksum gesture. By asking the user to perform the generated gesture, a secure channel can be authenticated. Results from a controlled user experiment (N = 13 participants, 1022 trials) demonstrate the feasibility of our technique, showing over 90% success rate in establishing a secure communication channel despite relying on complex gesture patterns. The authentication times of our method are over three-folds faster than with previous gesture-based solutions. The average execution time of a gesture is 5:7 seconds in our study, which is comparable to the input time of conventional text input based PIN authentication. Our approach is particularly well-suited for scenarios involving wearable devices that lack conventional input capabilities, e.g., pairing a smartwatch with an interactive display.

Locating points of interest (POIs) in cities is typically facilitated by visual aids such as paper maps, brochures, and mobile applications. However, these techniques require visual attention, which ideally should be on the surroundings. Non-visual techniques for navigating towards specific POIs typically lack support for free exploration of the city or more detailed guidance. To overcome these issues, we propose a multimodal, wearable system for alerting the user of nearby recommended POIs. The system, built around a tactile glove, provides audio-tactile cues when a new POI is in the vicinity, and more detailed information and guidance if the user expresses interest in this POI. We evaluated the system in a field study, comparing it to a visual baseline application. The encouraging results show that the glove-based system helps keep the attention on the surroundings and that its performance is on the same level as that of the baseline.

The user's understanding of information needs and the information available in the data collection can evolve during an exploratory search session. Search systems tailored for well-defined narrow search tasks may be suboptimal for exploratory search where the user can sequentially refine the expressions of her information needs and explore alternative search directions. A major challenge for exploratory search systems design is how to support such behavior and expose the user to relevant yet novel information that can be difficult to discover by using conventional query formulation techniques. We introduce IntentStreams, a system for exploratory search that provides interactive query refinement mechanisms and parallel visualization of search streams. The system models each search stream via an intent model allowing rapid user feedback. The user interface allows swift initiation of alternative and parallel search streams by direct manipulation that does not require typing. A study with 13 participants shows that IntentStreams provides better support for branching behavior compared to a conventional search system.

Peripheral physiological signals, as obtained using electrodermal activity and facial electromyography over the corrugator supercilii muscle, are explored as indicators of perceived relevance in information retrieval tasks. An experiment with 40 participants is reported, in which these physiological signals are recorded while participants perform information retrieval tasks. Appropriate feature engineering is defined, and the feature space is explored. The results indicate that features in the window of 4 to 6 seconds after the relevance judgment for electrodermal activity, and from 1 second before to 2 seconds after the relevance judgment for corrugator supercilii activity, are associated with the users' perceived relevance of information items. A classifier verified the predictive power of the features and showed up to 14% improvement predicting relevance. Our research can help the design of intelligent user interfaces for information retrieval that can detect the user's perceived relevance from physiological signals and complement or replace conventional relevance feedback.

Current search engines offer limited assistance for exploration and information discovery in complex search tasks. Instead, users are distracted by the need to focus their cognitive efforts on finding navigation cues, rather than selecting relevant information. Interactive intent modeling enhances the human information exploration capacity through computational modeling, visualized for interaction. Interactive intent modeling has been shown to increase task-level information seeking performance by up to 100%. In this demonstration, we showcase SciNet, a system implementing interactive intent modeling on top of a scientific article database of over 60 million documents.

This paper presents a pervasive adventure production called Lost Lab of Professor Millennium that experimented with different kinds of interaction techniques evaluating how they affected the adventure experience. The paper further reflects on the practical organization of the production targeted for schools and students between 12 and 15 years of age. Groups of up to four teenagers navigated through thirteen different kinds of pervasive computing experiences in checkpoints sharing a device providing augmented reality (AR) (MapLens) on a physical map and an adaptive marker-based AR guidance. Based on a Professor who lost her technologies, the story of the adventure provided a unifying narrative also through her fish Linus guiding groups through a variety of tasks in the checkpoints. The production was evaluated with direct observations, different kinds of video recordings, interviews and questionnaires. The evaluation revealed how groups shared the devices and performed collaborative interactions with the devices. The production received positive feedback from all stakeholders, but in terms of feasibility had some drawbacks. The evaluation indicated that the marker-based AR guidance techniques is practical, reliable and easy-to-use, and can be also used as a storytelling or story enhancing technique.

We present findings from a study of Tokens of Search, a system comprising physical RFID "tokens" that point to web content, and a wooden tray fixed to a small screen, which can be used to access that content. Three families lived with the system for a month, as an exploration of how tokens might be used as resources for practical action. Our findings highlight existing web practices and their individual and collective nuances; tokens were employed in the creation of short-term collections and long-lasting mementos, their physicality giving bookmarking a visibility that could be used to attract attention, serve as reminders, and make observable progress through tasks. However, while all families saw the potential for shared use, only one used it this way in earnest. We reflect on design choices that were expected to encourage collaboration, and the need to support key users such as parents when establishing joint practices.

Mobile devices have limited battery life, and numerous battery management applications are available that aim to improve it. This paper examines a large-scale mobile battery awareness application, called Carat, to see how it changes user behavior with long-term use. We conducted a survey of current Carat Android users and analyzed their interaction logs. The results show that long-term Carat users save more battery, charge their devices less often, learn to manage their battery with less help from Carat, have a better understanding of how Carat works, and may enjoy competing against other users. Based on these findings, we propose a set of guidelines for mobile battery awareness applications: battery awareness applications should make the reasoning behind their recommendations understandable to the user, be tailored to retain long-term users, take the audience into account when formulating feedback, and distinguish third-party and system applications.

Supporting exploratory search is a very challenging problem, not least because of the dynamic nature of the exercise: both the knowledge and interests of the user are subject to constant change. Moreover, whether the results for a query are informative is strongly subjective. What is informative to one user, is too specific for the other; specificity differs between users depending on their intent and accumulated knowledge about the domain.

Backchannels are tools that allow participants to discuss during a performance, such as lecture or presentation, without interrupting it. They are used in higher education and conferences to facilitate audience participation. This study examines backchannels in a Finnish primary school with a class of 12-13-year-old pupils. Backchannels can allow anonymous participation and this feature has been found practical in higher education. In this study, we observed that primary school pupils posted relevant messages at the same level as reported prior studies conducted in higher education. The pupils also appreciated the anonymity as it provided additional safety for self-expression.

Term-Relevance Prediction from Brain Signals (TRPB) is proposed to automatically detect relevance of text information directly from brain signals. An experiment with forty participants was conducted to record neural activity of participants while providing relevance judgments to text stimuli for a given topic. High-precision scientific equipment was used to quantify neural activity across 32 electroencephalography (EEG) channels. A classifier based on a multi-view EEG feature representation showed improvement up to 17% in relevance prediction based on brain signals alone. Relevance was also associated with brain activity with significant changes in certain brain areas. Consequently, TRPB is based on changes identified in specific brain areas and does not require user-specific training or calibration. Hence, relevance predictions can be conducted for unseen content and unseen participants. As an application of TRPB we demonstrate a high-precision variant of the classifier that constructs sets of relevant terms for a given unknown topic of interest. Our research shows that detecting relevance from brain signals is possible and allows the acquisition of relevance judgments without a need to observe any other user interaction. This suggests that TRPB could be used in combination or as an alternative for conventional implicit feedback signals, such as dwell time or click-through activity.

Interactive walk-up-and-use displays are spreading in a variety of settings where stand presentation situations are common. We contribute by characterizing a presentation situation and investigating specific design implications for presenters in this situation. We also introduce interface system that utilizes physics-modeled spherical content widgets for information browsing. The system includes dedicated features we developed to support presenters in content production and visualization. To investigate stand presentations and their support, we organized a field trial at an exhibition, collecting observational data from video analysis, interviews with presenters, and questionnaires from the audience and presenters. The field study confirms the importance of the presentation use case for public walk-up-and-use screens and points to dedicated design implications for simultaneous support for presenters and visitors, management of presentation territories, and personalization.

Interactive digital technologies pervade our shared spaces in personal, mobile, infrastructural and other embedded forms. These changes challenge the ways we understand and investigate the relationships between people, computing and settings. Responding to this situation—where ubiquitous computing is not only personal but also public, and where digital interactions may happen anywhere — this special issue explores how HCI research can use the strengths of an intersection of theory, practice and innovation in order to best address this conjunction of interactive technologies, public spaces and people interacting with or within both. \n \nWith this shift to interaction happening in public spaces, people can no longer be seen as just ‘users’ but have to be understood as acting assemblages of bodies and technologies in the space they inhabit. This shift highlights the need to understand users in every setting—including those at desks and at work—as socially and culturally situated humans with agency. While this is not a new research development, this special issue speculates on and discusses how the shift to ‘everywhere interactions’ makes it important for research in human–computer interaction to reach beyond existing conventions. With this work comes a need to understand the interacting human as a situated body that is at any moment also a possible spectacle: as a body, a human and a user who engages with technology in response to the situation while also being a ‘performance’ for others to witness. Understanding people as actors in their context allows us to regard every user as a ‘performing body’ and a ‘performing subject’, and it enables us to analyse the performative aspects of interaction in many other situations than those where we design for actually staged performances in a defined performance space or stage. In this sense, ‘performative interaction’ is both concerned with technologies for on-stage performers using technologies as part of artistic expression, and an analytical frame for understanding every situation in which people engage with technologies (in public space). The question becomes: where and in which situations is this analytical frame relevant and useful?

Web-based technologies are often built to capitalize on the flexibility and fluidity that is supported by the internet, with the value of 'access anywhere' underpinning a blurring of boundaries across home and work. Yet the home is well known in HCI to have a unique set of qualities that can use-fully be drawn upon when designing to support domestic life. In this paper we ask what it means to design domestic web-connected technologies, placing the aesthetic and material properties intrinsic to the home and home life at the centre of our design exploration. We present three concepts that were selected and prototyped from a broader process of research-through-design: Tokens of Search provides tangible handles to web resources; Hole in Space connects the home intimately to a remote place; and Manhattan enables the tangible exploration of events in the community, putting the home at the centre. Discussions in the paper consider not only how aesthetics is articulated in the material and digital properties of the artefacts, but also how a consideration of the properties of the home can create a potentially new design space to explore.

We introduce interactive intent modeling, where the user directs exploratory search by providing feedback for estimates of search intents. The estimated intents are visualized for interaction on an Intent Radar, a novel visual interface that organizes intents onto a radial layout where relevant intents are close to the center of the visualization and similar intents have similar angles. The user can give feedback on the visualized intents, from which the system learns and visualizes improved intent estimates. We systematically evaluated the effect of the interactive intent modeling in a mixed-method task-based information seeking setting with 30 users, where we compared two interface variants for interactive intent modeling, namely intent radar and a simpler list-based interface, to a conventional search system. The results show that interactive intent modeling significantly improves users' task performance and the quality of retrieved information.

Techniques for both exploratory and known item search tend to direct only to more specific subtopics or individual documents, as opposed to allowing directing the exploration of the information space. We present an interactive information retrieval system that combines Reinforcement Learning techniques along with a novel user interface design to allow active engagement of users in directing the search. Users can directly manipulate document features (keywords) to indicate their interests and Reinforcement Learning is used to model the user by allowing the system to trade off between exploration and exploitation. This gives users the opportunity to more effectively direct their search nearer, further and following a direction. A task-based user study conducted with 20 participants comparing our system to a traditional query-based baseline indicates that our system significantly improves the effectiveness of information retrieval by providing access to more relevant and novel information without having to spend more time acquiring the information.

Spaceify is a novel edge architecture and an ecosystem for smart spaces --- a technology that extends the mobile user view of today's common space services (e.g., WiFi) to a richer portfolio of space-centric, localized services and space-interactive applications.

Large interfaces are fixed to a certain use context, for example to a physical smart space. Mobile counterparts of public interfaces can allow user to continue interacting with the content also when leaving the space. However, wall applications make use of a large display surface and fitting the same user interface to the constraints of a mobile screen is challenging. Starting with Bubble Wall -- an information exploration application for multitouch walls -- we developed interfaces for browsing the same content graphs on mobile devices. A comparison study in exploration and navigation tasks was conducted with two mobile interfaces reproducing the large screen interactions: BubbleSpace with a more faithful redesign and BubblesDial reducing the interactions for better fit to a small screen. The BubblesDial scored significantly better in usability and performance evaluation, especially when priming with use of the Bubble Wall. We also present implications for the redesign of these large content graph interactions for mobile use.

Advanced interactive visualization such as in virtual environments and ubiquitous interaction paradigms pose new challenges and opportunities in considering real-time responses to subliminal cues. In this paper, we propose a synthetic reality platform that, combined with psychophysiological recordings, enables us to study in realtime the effects of various subliminal cues. We endeavor to integrate various aspects known to be relevant to implicit perception. The context is of consumer experience and choice of an artifact where the generation of subliminal perception through an intelligent 3D interface controls the spatio-temporal aspects of the information displayed and of the emergent narrative. One novel contribution of this work is the programmable nature of the interface that exploits known perceptive phenomena (e.g. masking, crowding and change blindness) to generate subliminal perception.

This paper reports the results of an exploratory study into inadvertent grip pressure changes on mobile devices with a focus on the differences between static lab-based and mobile walking environments. The aim of this research is to inform the design of more robust pressure input techniques that can accommodate dynamic mobile usage. The results of the experiment show that there are significant differences in grip pressure in static and walking conditions with high levels of pressure variation in both. By combining the pressure data with accelerometer data, we show that grip pressure is closely related to user movement.

ForcePhone is a mobile synchronous haptic communication system. During phone calls, users can squeeze the side of the device and the pressure level is mapped to vibrations on the recipient's device. The pressure/vibrotactile messages supported by ForcePhone are called pressages. Using a lab-based study and a small field study, this paper addresses the following questions: how can haptic interpersonal communication be integrated into a standard mobile device? What is the most appropriate feedback design for pressages? What types of non-verbal cues can be represented by pressages? Do users make use of pressages during their conversations? The results of this research indicate that such a system has value as a communication channel in real-world settings with users expressing greetings, presence and emotions through pressages.

In designing for engagement at a public multi-touch installation, we identified supporting multiple users and allowing for gradual discovery as challenges. In this paper, we present Worlds of Information, a multi-touch application featuring 3D Worlds, which provide access to different content. These 3D widgets gradually unfold and allow for temporal navigation of multimedia in parallel, while also providing a 2D plane where media can be shared. We report on a field trial at an exhibition using questionnaires and video ethnography. We studied engagement through questions adapted from Flow, Presence and Intrinsic Motivation questionnaires, which showed that users, overall, had a positive and social experience with the installation. The worlds effectively invited multiple users and provided for parallel interaction. While functionality was discovered gradually through social learning, the study demonstrates the challenges of designing multi-touch applications for walk-up-and-use displays.

We present findings from field trials of MapLens, a mobile augmented reality (AR) map using a magic lens over a paper map. Twenty-six participants used MapLens to play a location-based game in a city centre. Comparisons to a group of 11 users with a standard 2D mobile map uncover phenomena that arise uniquely when interacting with AR features in the wild. The main finding is that AR features facilitate place-making by creating a constant need for referencing to the physical, and in that it allows for ease of bodily configurations for the group, encourages establishment of common ground, and thereby invites discussion, negotiation and public problem-solving. The main potential of AR maps lies in their use as a collaborative tool.

Mobile devices such as the iPhone provide state of the art interaction capabilities also for web browser applications. Our mobile development is targeted to a Energy Awareness application that provides playful access to detailed and realtime information on energy consumption of appliances of a household. Using the available Safari Browser that adopts W3C web standards we demonstrate the implementation of a 3D carousel giving access to cards on a web page where each card gives access to information on one appliance. The carousel can be browsed using the multitouch capability of the iPhone. We describe the programming approach and discuss the lesson learned in developing the touch interaction with the carousel.

Recently, multimodal and affective technologies have been adopted to support expressive and engaging interaction, bringing up a plethora of new research questions. Among the challenges, two essential topics are 1) how to devise truly multimodal systems that can be used seamlessly for customized performance and content generation, and 2) how to utilize the tracking of emotional cues and respond to them in order to create affective interaction loops. We present PuppetWall, a multi-user, multimodal system intended for digitally augmented puppeteering. This application allows natural interaction to control puppets and manipulate playgrounds comprising background, props, and puppets. PuppetWall utilizes hand movement tracking, a multi-touch display and emotion speech recognition input for interfacing. Here we document the technical features of the system and an initial evaluation. The evaluation involved two professional actors and also aimed at exploring naturally emerging expressive speech categories. We conclude by summarizing challenges in tracking emotional cues from acoustic features and their relevance for the design of affective interactive systems.

We present data from detailed observations of CityWall, a large multi-touch display installed in a central location in Helsinki, Finland. During eight days of installation, 1199 persons interacted with the system in various social configurations. Videos of these encounters were examined qualitatively as well as quantitatively based on human coding of events. The data convey phenomena that arise uniquely in public use: crowding, massively parallel interaction, teamwork, games, negotiations of transitions and handovers, conflict management, gestures and overt remarks to co-present people, and "marking" the display for others. We analyze how public availability is achieved through social learning and negotiation, why interaction becomes performative and, finally, how the display restructures the public space. The multi-touch feature, gesture-based interaction, and the physical display size contributed differentially to these uses. Our findings on the social organization of the use of public displays can be useful for designing such systems for urban environments.

This paper seeks to develop a better understanding of the contribution of materiality for creativity in collaborative settings, exploring the ways in which it provides resources for persuasive, narrative and experiential interactions. Based on extensive field studies of architectural design workplaces and on examples from art works, we show: how the variety of material features expands communicative resources and provide border resources for action, in their peripheral, evocative, and referential function; how spatiality supports the public availability of artefacts as well as people's direct, bodily engagement with materiality; and finally how materiality is part of performative action, looking at temporal frames of relevance and emergence in specific events. We conclude with implications for the development of novel interface technologies.

Previous attempts to support spectators at large-scale events have concentrated separately on real-time event information, awareness cues, or media-sharing applications. CoMedia combines a group media space with event information and integrates reusable awareness elements throughout. In two field trials, one at a rally and the other at a music festival, we found that CoMedia facilitated onsite reporting to offsite members, coordination of group action, keeping up to date with others, spectating remotely, and joking. In these activities, media, awareness cues, and event information were often used in concert, albeit assuming differing roles. We show that the integrated approach better supports continuous interweaving of use with the changing interests and occurrences in large-scale events.

Most large public displays have been used for providing information to passers-by with the primary purpose of acting as one-way information channels to individual users. We have developed a large public display to which users can send their own media content using mobile devices. The display supports multi-touch interaction, thus enabling collaborative use of the display. This display called CityWall was set up in a city center with the goal of showing information of events happening in the city. We observed two user groups who used mobile phones with upload capability during two large-scale events happening in the city. Our findings are that this kind of combined use of personal mobile devices and a large public display as a publishing forum, used collaboratively with other users, creates a unique setting that extends the group's feeling of participation in the events. We substantiate this claim with examples from user data.

Traditionally, mobile media sharing and messaging has been studied from the perspective of an individual author making media available to other users. With the aim of supporting spectator groups at large-scale events, we developed a messaging application for camera phones with the idea of collectively created albums called Media Stories. The field trial at a rally competition pointed out the collective and participative practices involved in the creation and sense-making of media, challenging the view of individual authorship. Members contributed actively to producing chains of messages in Media Stories, with more than half of the members as authors on average in each story. Observations indicate the centrality of collocated viewing and creation in the use of media. Design implications include providing a ""common space"" and possibilities of creating collective objects, adding features that enrich collocated collective use, and supporting the active construction of awareness and social presence through the created media.

As Brenda Laurel noted as far back as 1992, the operation of computers has always been a performative activity (Laurel, 1992). A system’s state changes as a computer runs through a program acting out the tasks specified in the script of a program. With interactive systems, human actors take their place on stage alongside computers, performing activities with and through such systems. The recent emergence of ubiquitous and tangible computing moves the stage of the interaction from the virtuality of the screen to the physical environment. This provides opportunities to address performative interactions that include bodily movements to create novel multimodal approaches. For interaction designers, this requires thinking about interaction in a different way, for example considering the role of the body, beyond ergonomics, for its increased relevance as a presentational, representational and experiential medium. Recently there has been a growing interest in developing interaction design methods that more explicitly recognise and exploit the performative elements and potentials of design activity itself.

Interesting characteristics of large-scale events are their spatial distribution, their extended duration over days, and the fact that they are set apart from daily life. The increasing pervasiveness of computational media encourages us to investigate such unexplored domains, especially when thinking of applications for spectator groups. Here we report of a field study on two groups of rally spectators who were equipped with multimedia phones, and we present a novel mobile group media application called mGroup that supports groups in creating and sharing experiences. Particularly, we look at the possibilities of and boundary conditions for computer applications posed by our findings on group identity and formation, group awareness and coordination, the meaningful construction of an event experience and its grounding in the event context, the shared context and discourses, protagonism and active spectatorship. Moreover, we aim at providing a new perspective on spectatorship at large scale events, which can make research and development more aware of the socio-cultural dimension.

Camera phones have been viewed simplistically as digital cameras with poor picture quality while neglecting the utility of the two key functionalities of mobile phones: network connection and access to personal information. This is the first HCI paper to examine mobile photos from a systemic perspective: how assignment of phases of mobile photo lifecycle to different platforms affects social discourse around shared photos. We conducted a 6-week user trial of MobShare, a tripartite system with dedicated functions and task couplings for a mobile phone, a server, and a PC browser. We analyze how MobShare's couplings and distribution of functionalities affected the observed types of social discourse that formed around mobile photos: in-group post-event discourse, self-documents and reports, greetings and thanks. Several central design issues arising from the systemic view are discussed: heterogeneity of environments, integration and distribution of functionalities, couplings and decouplings of interaction tasks, notification mechanisms, and provision of necessary UI resources for different tasks.

The wide availability of digital recording devices leads to investigate how multimedia content can be navigated beyond a desktop computer set up. We present a system and a variety of applications, to navigate multimedia recordings from visits making use of: large multiple projections, location information to organise media and re-experience aspects of visits, physical interfaces as gesture-based interaction and other interfaces that render media more tangible and therefore more readily available. Analysing field trials we discuss the expressiveness and experiential aspects of gesture based interfaces as important features in navigating multimedia in "immersive" environments.

The paper contrasts two views on knowing: those of the observer and the active actor in a situation. The paper suggests that there are design cases where performance can produce different knowledge. The paper reviews the use of performances in theatre and discusses a technique to use performances in the design of mobile services. The technique is illustrated by example. The session are analysed to describe creativity and knowledge of participants.