As smart technologies become increasingly embedded in daily life, the demand for natural and efficient human-machine interaction continues to grow. Multimodal systems, which allow interacting with multiple modalities, such as speech or gestures, potentially supporting a more natural and efficient way for users to communicate with machines. However, the complexity of these systems introduces significant challenges in development and evaluation. Nevertheless, even when interaction data are collected, these systems can generates large volumes of data, making it difficult to understand how users engage with the system, how the system responds, identify system limitations as well as uncover interaction patterns. In this sense, visualisation can play a crucial role in helping researchers and developers make sense of multimodal interaction data in order to reveal communication patterns, uncover system bottlenecks, and support deeper insight into system (and user) behavior. In this work we explore the potential of interactive visualisation, through a case study considering a multimodal system, and demonstrate how interactive visualisation can be useful to respond to a set of questions that we want to know about the system evaluation.

This work explored the impact of expert representation during the (re)use of Augmented Reality (AR) instructions for remote assistance.A user study with sixteen participants compared three conditions: C1-no agent, C2-human agent, and C3-virtual agent, within a real-world maintenance scenario.Results indicate that reusing AR-instructions was well accepted, even when not perfectly aligned with the task context.While C2 reported higher levels of social presence and trust, C3 was considered the best option when reusable content was used and limited interaction was required.This allowed to focus more on the task and less on the agent representation.

The aging population creates the need to promote and encourage healthy and active lifestyles to help prevent chronic diseases or injuries and consequently alleviate the pressure on healthcare systems. Technological advancements present an opportunity to monitor people more continuously and provide them with useful and relevant information that can indicate potential problems or simply help them to stay or be more active and healthy. Smart mirrors offer the promising capability of monitoring individuals daily, in their own homes, without interfering with their usual routine, a key factor for user acceptance and widespread adoption. To obtain an overview of the state-of-the-art on smart mirrors for health/well-being monitoring, we performed a review of the literature on the topic, focusing on mirrors incorporating sensing capabilities. Relevant information was then extracted from the articles found and a critical analysis was carried out. Based on the results of this analysis, as well as our own experience, we provide recommendations we consider important to take into account in the future development of smart mirrors that monitor people's health and well-being.

Abstract Remote collaboration using eXtended Reality (XR) has been explored to establish a common ground between physically distributed individuals. To achieve usable and impactful solutions, it is paramount to understand how different content creation and display methods contribute to the work effort of each member (remote and on-site). Additionally, explore how various task scenarios influence the collaborative process, specifically examining how activities with unique characteristics and complexities affect remote work. Hence, the question arises, ’How do different display and interaction methods impact the collaborative process within specific task domains?’ In this paper, two user studies with 30 distinct participants each are described, focusing on different content creation support (Laptop Computer; Video Wall & Keyboard; Interactive Projector) and display methods (Hand-Held Device (HHD); HHD & Articulated Support; Head-Mounted Display (HMD)) for remote and on-site collaborators respectively, during scenarios of remote guidance. Plus, different physical tasks were considered for each study (Lego pieces assembly; Tangram puzzle assembly; Maintenance procedures; Resource management in a map; Training activity in a laboratory classroom), to understand which method stands out according to the characteristics of the said tasks. We report the results obtained, suggesting that for the remote part of the collaborative process, using a Video Wall & Keyboard appears to be more suitable for tasks associated with maintenance procedures, learning, and training activities. As for the on-site part, using HMD was considered the better option for maintenance, and learning tasks, closely followed by HHD & Articulated Support.

People with language impairments can have difficulties expressing themselves to others, leading to major limitations to their safety, independence, and quality of life in general. Aphasia is an example of an acquired language impairment that affects many people (around 2 million in the United States), being commonly caused by stroke, but also by other brain injuries. Several augmentative and alternative communication solutions are available to help people with communication difficulties, but they are generally not suitable for all contexts of use (e.g., lying in bed). In the scope of the “APH-ALARM” project, which aimed at developing solutions to support people with Aphasia, we envision a system for the bedroom that enables conveying messages to be sent to a caregiver or relative, for example. Focusing on gesture input, in this contribution, we investigated if smartwatch sensors and machine learning (ML) can be used to recognise arm gestures executed while lying. We explored different factors, namely the feature set, size of the sliding window used for feature extraction, and ML classifier. The results obtained with data gathered from ten subjects are promising, with the best factor combinations for the user-independent solution leading to a mean macro F1 score of 94% or 95%. They demonstrate the potential of using wearables to develop a gesture input modality for the in-bed scenario, which can also potentially be extended to other contexts (e.g., sitting in a bed, chair, or sofa, or standing). This research also provides useful insights that inform future work, including the development and deployment of communication support systems that can benefit not only people with communication difficulties (e.g., more independence), but also those caring for them (e.g., more peace of mind). • A new gesture-based interaction modality for communication support is proposed. • The proposal is aligned with the needs of users with Aphasia while lying in bed. • The use of wearable sensors for arm gesture recognition is investigated. • The best achieved mean F1 score was of 95%, for a user-independent solution.

The environments we live in are becoming increasingly smarter, creating the need of finding suitable ways to interact with them. While interaction modes such as speech, text, and touch remain prevalent, there are associated challenges related to background noise interference, accessibility issues, and reliance on devices (e.g., mobile, wearable). Gesture-based interaction emerges as a promising alternative or complement to those modes. In this context, radars represent a good option for enabling gesture recognition, since they rely on radio waves to detect moving targets, not presenting the disadvantages of other sensors such as cameras and wearables, which can be considered too intrusive by the users. This contribution intends to investigate the capabilities of radars for gesture-based interaction in smart environments, focusing on the possibility of using data provided by a radar and transfer learning to recognize a set of pre-defined arm gestures regardless of the distance between the user and radar. The results obtained based on data collected from a subject standing at four different distances (1, 2, 3, and 4 m) from the radar, showed a poor performance when considering a distance-independent solution. For a distance- dependent solution, relatively good results were achieved when considering all four distances together (mean accuracy of 96 %). These results are very useful for future developments of gesture- based interaction with smart environments in real scenarios.

People with aphasia (PwA) face a set of communication barriers in several contexts, e.g., in social contexts. Since augmentative and alternative communication (AAC) tools are often too general and provide limited support to this audience, the aim of this work is to explore novel ways of supporting communication for PwA that take into account their needs and characteristics and move beyond the scope covered by current solutions. This work describes the design and development of the first proof-of-concept of Aphluentia, a tool to support communication in everyday social situations for people with fluent aphasia, designed by working with speech and language therapists with experience working with PwA to inform and evaluate its characteristics. At its current stage, Aphluentia establishes the grounds for discussion and assessment of the proposal by PwA.

Most efforts on Augmented Reality (AR) remote collaboration have been devoted to on-site individuals in need of assistance. Yet, since remote experts do not have access to the local environment, it is also important to explore how different methods affect the situation understanding and interaction. This work describes a user study with 16 participants aimed at comparing how remote individuals interact with standard (C1) and large-scale displays (C2) while providing assistance during a remote maintenance scenario. Results suggest condition C2 was preferred by the majority of participants, being considered more useful to analyze important details of the task context, as well as more easily create content.

Although Mixed Reality (MR) has been explored to support scenarios of remote collaboration, the majority of studies reported in literature have focused on synchronous use-cases, where all team-members can interact in real-time. Hence, an opportunity exist to explore asynchronous situations, in which collaborative actions take place at different times. Despite the physical distance and different time zones, remote experts can generate MR-instructions to assist with emerging problems, suggesting to on-site collaborators where to act, and what to do. This work explored three distinct notification methods for such scenarios: C1 - visual cue; C2 - audio cue; C3 - tactile cue, aiming to comprehend which one captures the attention of on-site team-members easily. A user study with 12 participants was conducted to evaluate these conditions during an asynchronous scenario of remote maintenance. We report participants insights, suggesting tactile cues represent the predominant condition given the scenario context.

To support the nuances of collaborative work, many researchers have been exploring the field of Augmented Reality (AR), aiming to assist in co-located or remote scenarios. Solutions using AR allow taking advantage from seamless integration of virtual objects and real-world objects, thus providing collaborators with a shared understanding or common ground environment. However, most of the research efforts, so far, have been devoted to experiment with technology and mature methods to support its design and development. Therefore, it is now time to understand where the field stands and how well can it address collaborative work with AR, to better characterize and evaluate the collaboration process. In this article, we perform an analysis of the different dimensions that should be taken into account when analysing the contributions of AR to the collaborative work effort. Then, we bring these dimensions forward into a conceptual framework and propose an extended human-centered taxonomy for the categorization of the main features of Collaborative AR. Our goal is to foster harmonization of perspectives for the field, which may help create a common ground for systematization and discussion. We hope to influence and improve how research in this field is reported by providing a structured list of the defining characteristics. Finally, some examples of the use of the taxonomy are presented to show how it can serve to gather information for characterizing AR-supported collaborative work, and illustrate its potential as the grounds to elicit further studies.

Augmented Reality (AR) has been explored with the objective to assist in scenarios of co-located or remote collaboration. To help understand how well collaborative work can be addressed with AR, it is important to foster harmonization of perspectives and create a common ground for systematization and discussion. In this vein, understand relationships among existing dimensions of collaboration, as well as identify research opportunities, is of paramount importance and thus tools that allow visually exploring information associated with Collaborative AR may be most valuable. In this paper, we present a first effort towards the creation of such an interactive visualization tool for exploration and analysis of collaborative AR research. It allows visualize data of selected papers organized according to a human-centered taxonomy on collaborative AR. In order to get insights into whether the structure was understood and if the representation was clear and efficient to use, we evaluated the proposed tool through a user study with 40 participants. Results suggest the tool has potential towards the creation of a shared understanding and identification of existing patterns, trends and opportunities within the field of collaborative AR.

Gaze information has the potential to benefit Human-Computer Interaction (HCI) tasks, particularly when combined with speech. Gaze can improve our understanding of the user intention, as a secondary input modality, or it can be used as the main input modality by users with some level of permanent or temporary impairments. In this paper we describe a multimodal HCI system prototype which supports speech, gaze and the combination of both. The system has been developed for Active Assisted Living scenarios.

Left ventricle functional analysis can be performed using coronary CT angiography (CTA) images. In recent years it has been proposed that CTA imaging might also be used for myocardial perfusion assessment, by visually looking for hypoattenuated regions that may correspond to hypoperfusion. Assigning a detected lesion to a specific myocardial segment requires significant effort while the clinician matches the American Heart Association (AHA) standard myocardial segments to the anatomy of each patient. This article presents an interactive tool which, based on the existence of a previous left ventricle segmentation, automatically matches the lesions detected by the clinician to a myocardial segment and provides feedback over the image regarding the myocardial segments with detected lesions and their severity.

Visualizing large family structures is becoming increasingly important, as more genealogical data becomes available. A space-filling h-tree layout pedigree has been recently proposed to make better use of the available space than traditional representations. In a previous paper we applauded the technique's usage of available space but remarked that it makes generation identification difficult and does not allow navigating to descendants of represented individuals. A set of extensions was proposed to help overcome these limitations and a preliminary evaluation suggested that those extensions enhance the original technique. This paper presents a more thorough evaluation carried out to assess if and how the proposed extensions improve the original h-tree layout pedigree technique. Results suggest that these extensions improve user performance on some tasks, effectively provide new functionality, and generally enhance user experience.

Abstract Coronary CT angiography is widely used in clinical practice for the assessment of coronary artery disease. Several studies have shown that the same exam can also be used to assess left ventricle (LV) function. Even though coronary CT angiography provides data concerning multiple cardiac phases, along the cardiac cycle, LV function is usually evaluated using just the end‐systolic and end‐diastolic phases. This unused wealth of data, mostly due to its complexity and the lack of proper tools, has still to be explored to assess if further insight is possible regarding regional LV functional analysis. Furthermore, different parameters can be computed to characterize LV function and though some are well known by clinicians others still need to be tested concerning their value in clinical scenarios. Based on these premises, we present several parameters characterizing global and regional LV function, computed for several cardiac phases over one cardiac cycle. The data provided by the computed parameters is shown using a set of visualizations allowing synchronized visual exploration of the different data. The main purpose is to provide means for clinicians to explore the data and gather insight over their meaning and their correlation with each other and with diagnosis outcomes.

This paper presents a user study performed to compare the usability of the Wiimote as an input device to visualize information and navigate in Google Earth using two different configurations. This study had the collaboration of 15 participants which performed a set of tasks using the Wiimote as an input device while the image was projected on a common projection screen, as well as a mouse on a desktop. Results show that most users clearly preferred one of the Wiimote configurations over the other, and over the mouse; moreover, they had better performances using the preferred configuration, and found it easier to use.

Cardiac angiography using multiple detector row computerized tomography (MDCT) scanners provides 3D data concerning the heart and, in particular, the left ventricle (LV), for different cardiac phases along one cardiac cycle. Exploring this data for LV function analysis is not an easy task, given the amount of data and time involved in segmenting (or revising results provided by automatic segmentation methods) up to 12 cardiac phases. CardioAnalyser, a tool for 4D LV segmentation from MDCT data which provides a protocol to help perform LV segmentation of all cardiac phases available is presented. It uses an automatic segmentation algorithm along with a procedure which guides the user through the process. Its main goal is to reuse as much information as possible from one cardiac phase to the next in order to reduce segmentation time and the amount of user interaction.

The appropriate use of color in Visualization is a very important subject. The choice of the proper color scale to use with a particular data set is not just a matter of choosing the prettiest representation. Throughout the years researchers have studied this subject and managed to propose guidelines which help users along the process of color scale selection. This article presents a brief overview on the subject focusing on the desired properties for color scales, the guidelines that should drive their choice, the advantages of applying those guidelines, the experimental research work on the field, and the tools proposed to help non-expert users.

Polygonal meshes are used in many areas to model different objects and structures. Depending on their applications, they sometimes have to be processed to, for instance, reduce their complexity (simplification). This mesh processing introduces error, whose evaluation is important when choosing the kind of processing that is to be done for a particular application. Although some mesh comparison tools are described in the literature, little attention has been given to the way results are presented. A tool is presented which enhances the way users perform mesh analysis and comparison, by providing an environment where several visualization options are available and can be used in a coordinated way.