International audience

As a field, haptics encompasses a wide and diverse array of technologies, modalities, and applications, which makes it challenging to understand, classify, and compare comprehensively. In this paper, we introduce Haptree: a design space framework developed based on a review of existing literature and building on top of existing taxonomies. Unlike other haptic frameworks that focus on specific aspect of haptics, Haptree is intended to show the big picture of haptic experiences, connecting together aspects of haptic experience that used to be isolated in other frameworks. Haptree maps haptic experiences in five different stages, from hardware characteristics to end-user applications. It provides practical guidelines for designing new haptic experiences by referencing existing approaches and use cases, or pointing to a current gap in the literature. Furthermore, Haptree is intended to be a community-driven tool, designed for iterative refinement and contribution by the haptic research community. Finally, we outline future steps, including validating and enhancing Haptree through community engagement and empirical studies.

Since its early development in the 1990s, Virtual Reality (VR) technology, particularly head-mounted displays (HMDs), has seen significant advancements. In 1999, an empirical study demonstrated that passive haptics could significantly improve both user performance and preference in 2D tasks. In this paper, we replicate this experiment using modern VR hardware to investigate the influence of technological evolution on the relevance of passive haptics in similar scenarios. Our findings show that, for the tasks examined, performance in non-haptic conditions with current VR systems is comparable to that in haptic conditions from 1999, challenging the relevance of passive haptics for such tasks for nowadays standards. Our results imply that enhancements in visual fidelity, tracking and interaction design may have reduced the performance gap that passive haptics were previously used to address.

This article describes a tangible interface for an e-coach, co-designed in four countries to meet older adults’ needs and expectations. The aim of this device is to coach the user by giving recommendations, personalized tasks and to build empathy through vocal, visual, and physical interaction. Through our co-design process, we collected insights that helped identifying requirements for the physical design, the interaction design and the privacy and data control. In the first phase, we collected users’ needs and expectations through several workshops. Requirements were then transformed into three design concepts that were rated and commented by our target users. The final design was implemented and tested in three countries. We discussed the results and the open challenges for the design of physical e-coaches for older adults. To encourage further developments in this field, we released the research outputs of this design process in an open-source repository.

Practical simulation is increasingly used to develop reasoning skills during learning. The analysis of the scene and the correct execution of actions require an awareness of the situation and the activities performed by the student. Eye-tracking feedback (i.e., a video recording of the practical simulation with an overlay of the gaze point) can allow students and teachers to enhance the skills of analysis and execution of the practical activities performed. In this article, we present the implementation of an innovative pedagogical process for nursing students in Switzerland. It involves the use of eye-tracking glasses to improve learning through the enhancement of metacognition after a simulation. The results of a first test session done with 15 undergraduate students are reported.

In a world where technological advancements are progressing at a vertiginous pace, social networks, online games, virtual worlds, streaming services, and remote work are part of everyday life. This is the case for the work environment, with the use of technological tools and home offices. In contrast, harmful aspects have been amplified, such as stress that affects occupational health. Lately, considerable interest has been gained in the affective domain in improving the occupational situation using empathic responses. In this work, we study the effect of machine empathic responses such as blue light, relaxing music, and the combination of light and music on people performing stressful tasks in an occupational environment. Thirty five participants tested different stimuli, eleven tested the music condition, twelve the light effect, and another twelve the combination of light and music. The monitoring of the heart rate variability along with psychological measures show that empathic responses can help reduce humans stress levels.

The communication between humans and artificial agents is becoming crucial and significant in daily life, especially with the advancements in the fields of human-robot and human-computer interaction. For these artificial agents to be recognized as social beings, they should exhibit emotional and empathic behaviors. However, there is no global agreement on measuring the empathic capabilities of these agents. For this reason, the scientific community has paid a significant focus on developing a standardized metric to perceive artificial agents' empathy. In this regard, this article provides a discussion on challenges in artificial empathy evaluation and researches the developments to discuss the factors and recommendations to design a globally accepted metric. It also discusses the qualities required for a globally accepted and standardized metric. Finally, an adaptation to an existing questionnaire is proposed for the evaluation of empathy in artificial agents.

Conversational agents are getting increasingly popular and find applications in health and customer services. Conversations in these fields are often emotionally charged. It is, therefore, necessary to handle the conversation with some degree of empathy to be effective. In this work, we leverage advances in the field of natural language processing to create a dialogue system that can convincingly generate empathic responses to text-based messages. To improve the system's ability to converse with empathy, we train the language model on empathic conversations and inject additional emotional information in the response generation. We propose two chatbots: a benchmark bot and an empathic bot. Additionally, we implement an emotion classifier that allows us to predict the emotional state of text-based messages. We evaluate both chatbots in quantitative studies and compare them with human responses in qualitative studies involving human judges. Our evaluation shows that our empathic chatbot outperforms the benchmark bot and even the human-generated responses in terms of perceived empathy. Additionally, we achieve state-of-the-art results in terms of response quality using transformer-based language models. Finally we report that we can double the initial performance of the emotion classifier using undersampling techniques, yielding a final F1-score of 0.81 in six basic emotions.

Search engines are among the most popular web services on the World Wide Web. They facilitate the process of finding information using a query-result mechanism. However, results returned by search engines contain a lot of duplications. For instance, when a user searches for a query q on Google, he will get n number of links divided into m number of pages. Although returned links have different content, exactly similar content is still existing on different links returned by search engines. This problem is referred to as duplication. Solving this problem, will increase the quality of search results as well as reducing the time search per query. In this paper, we introduce a new method called REDUCE (REsult DUplication detection in searCh Engines), to address this problem. It implements a semi-supervised approach. It approximately measures the similarity between the web pages and we suggest a new method to group the search results based on their similarity. To evaluate our method, we collect data from Google and other search engine platforms. We show that our method can solve this problem on different search engine platforms with different languages. We empirically evaluated our results on different classification algorithms and reached an accuracy of 96.7%.

In conditionally automated driving, drivers do not have to constantly monitor their vehicle but they must be able to take over control when necessary. In this paper, we assess the impact of instructions about limitations of automation and the presentation of context-related information through a mobile application on the situation awareness and takeover performance of drivers. We conducted an experiment with 80 participants in a fixed-base driving simulator. Participants drove for an hour in conditional automation while performing secondary tasks on a tablet. Besides, they had to react to five different takeover requests. In addition to the assessment of behavioral data (e.g. quality of takeover), participants rated their situation awareness after each takeover situation. Instructions and context-related information on limitations combined showed encouraging results to raise awareness and improve takeover performance.

Humans nowadays are tending to spend too much time in front of their screens. Direct interaction between humans is falling in numbers and people are losing their empathic behaviour. By integrating empathy and emotions in everyday objects researchers can address this problem. In addition we can have a positive effect on our lives, from physical and mental health through tackling many issues, like productivity, time wasting, stress and other problems. In this article, we tackle the productivity problem by presenting the Empathic Flower Companion (EFC) that will be using the expression of emotions to help the human through their working day. It will be monitoring their time and at the same time analysing the websites they will be surfing. The concept proposed will reduce the time wasted on unproductive websites. The results show an increase of 15% in the productivity of the testers, which shows that EFC was effective in reducing the amount of time wasted.

In 2017, Vaswani et al. proposed a new neural network architecture named Transformer.That modern architecture quickly revolutionized the natural language processing world.Models like GPT and BERT relying on this Transformer architecture have fully outperformed the previous state-of-theart networks.It surpassed the earlier approaches by such a wide margin that all the recent cutting edge models seem to rely on these Transformer-based architectures.In this paper, we provide an overview and explanations of the latest models.We cover the auto-regressive models such as GPT, GPT-2 and XLNET, as well as the auto-encoder architecture such as BERT and a lot of post-BERT models like RoBERTa, ALBERT, ERNIE 1.0/2.0.

T e-Coaches are promising intelligent systems that aims at supporting human everyday life, dispatching advices through different interfaces, such as apps, conversational interfaces and augmented reality interfaces. This workshop aims at exploring how e-coaches might benefit from spatially and time-multiplexed interfaces and from different communication modalities (e.g., text, visual, audio, etc.) according to the context of the interaction.

Is it helpful for a medical physical health chatbot to show empathy? How can a chatbot show empathy only based on short-term text conversations? We have investigated these questions by building two different medical assistant chatbots with the goal of providing a diagnosis for physical health problem to the user based on a short conversation. One chatbot was advice-only and asked only the necessary questions for the diagnosis without responding to the user's emotions. Another chatbot, capable of showing empathy, responded in a more supportive manner by analyzing the user's emotions and generating appropriate responses with a high empathic accuracy. Using the RoPE scale questionnaire for empathy perception in a human-robot interaction, our empathic chatbot was rated significantly better in showing empathy and was preferred by a majority of the preliminary study participants (N=12).

In the context of highly automated driving, the driver has to be aware of driving risks and to take over control of the car in hazardous situations. The goal of this paper is to categorize and analyze the factors that lead to such critical scenarios. To this purpose, we analyzed limitations of Advanced Driver-Assistance Systems (ADAS) extracted from owner manuals of 12 partially automated cars available on the market. A taxonomy with 6 macro-categories and 26 micro-categories is proposed to classify and better understand the limitations of these vehicles. We also investigated if these limitations are conveyed to the driver through Human-Machine Interaction (HMI) in the car. Some suggestions are made to better communicate these limitations to the driver in order to raise his/her situation awareness.

This paper presents the decision support system that has been defined and developed under the umbrella of the NESTORE project. The main goal of the proposed system is to help users in selecting coaching plans by proposing personalised recommendations based on their behaviours and preferences. Recognising such behaviours and their evolution over time is therefore a crucial element for tailoring the interaction of the system with the user. A three-layer system composed of pathways, coaching activity plans, and coaching events, constitutes the so-called coaching timeline on which the analysis is grounded. Various techniques are used to model and personalise the recommendations and feedback. Firstly, the indicators are extracted from disparate data sources, then these are modelled through a profiling system and, finally, recommendations on the pathways and coaching plans are performed through a scoring and a tagging system.

We introduce a new dataset, dubbed UNICITY1, for the task of detecting people in security airlocks in top view depth images. If security companies have been relying on computer systems and algorithms for a long time, very few are trusting artificial intelligence and more specifically machine learning approaches in production environments. We are confident that the recent advances in these domains, especially with the democratization of deep learning, will open new horizons for security systems. We release this dataset to encourage the development of such approaches in the scientific community.UNICITY consists of 58k images collected from 65 recorded sequences with one or two people performing different behaviors including attacks and trickeries (e.g. tailgating2). It also provides full annotation of people such as the location of head and shoulders. As as result, UNICITY is perfectly suited for training and adapting machine learning algorithms for video surveillance applications. This paper presents the data collection, an evaluation protocol, as well as two baseline methods for attack detection.

This article presents the first step of a project focusing on enhancing the management of bike-sharing systems. The objective of the project is to optimize the daily rebalancing operations that need to be performed by operators of bike-sharing systems using machine-learning algorithms and constraint programming. This study presents an evaluation of machine learning algorithms developed for forecasting the availability of bikes on three Swiss bike-sharing networks. The results demonstrate the superiority of the Multi-Layer Perceptron algorithm for forecasting available bikes at station-level for different prediction horizons and its applicability for real-time prediction generation.

Current interactions for the Internet of Things are often constrained behind a screen. With the Internet of Tangible Things (IoTT) we aim at promoting the design of richer interactions, embodied in physical IoT objects. To this purpose, we propose a card set for the design of tangible interaction with IoT objects, which contains 8 cards for tangible interaction properties and 8 for IoT properties, in order to explore how tangible properties can be exploited for enhancing the interaction with IoT objects. We tested the card set in a dedicated workshop, observing that participants were able to explore most of the tangible and IoT properties. To complement the IoT card set, a hardware prototyping toolkit with examples for each of the 8 tangible properties is currently under development.

There is an increasing interest in the HCI research community to design richer user interactions with the Internet of Things (IoT). This studio will allow exploring the design of tangible interaction with the IoT, what we call Internet of Tangibles. In particular, we aim at investigating the full interaction-attention continuum, with the purpose of designing IoT tangible interfaces that can switch between peripheral interactions that do not disrupt everyday routines, and focused interactions that support user's reflections. This investigation will be conducted through hands-on activities where participants will prototype tangible IoT objects, starting by a paper prototyping phase, supported by design cards, and followed by an Arduino prototype phase. The purpose of the studio is also establishing a community of researchers and practitioners, from both academy and industry, interested in the field of tangible interaction with the Internet of Things.

Many authors can share the same name and this constitutes a serious problem that affects the relevancy of retrieval results and constitutes our motivation of finding such approach to cover this issue at the author names entity level. Solving such a problem may return with positive gain at the level of document retrieval, web search and the quality of data. This entity resolution task can be tackled as an unsupervised problem, where there are set of features that can be employed for the resolution job, or as supervised problem to compute the similarities among two citations and then classify if they are the same or not. Recent approaches usually utilize features such as: co-author, venue, topic similarity, affiliations and title of publications to deal with author ambiguity. In this paper, three attributes are used to treat this problem sequentially. The co-authorship firstly which is a well-known attribute, and then the topic and affiliation extracted from biographies, which can be found inside the publication, and this is our novelty frame in this paper.

Exploiting facts that are published online using semi-structured or unstructured formats is a highly complex task, pieces of data are typically published in isolation and periodically updated in a bulk fashion without any coordination. In this paper, we propose a new software pipeline tackling this issue and semantically lifting online facts as Linked Data in a continuous manner. Our solution, LinkedPolitics, extracts, converts, interlinks, and finally makes available as Linked Data online facts that could not be transparently exposed and exploited otherwise. Our system is online, in the sense that it automatically makes available new facts as they are uploaded online. We also present a deployment of LinkedPolitics centered around the Swiss Parliament. Our deployment extracts and semantically lifts a flurry of political facts that are periodically published in unstructured form by the Library Service of the Swiss Federal Assembly. It then automatically materializes a number of views to support different visualizations of the data in order to capture the actions, announcements or votes made by the politicians, as well as their relationships to each other and to external entities such as companies.

This paper presents EmotiPlant, a system that aims to facilitate the nurturing of indoor plants for older adults. The proposed concept exploits the idea of a plant that is able to express emotions and display its status in relation to the environmental conditions. Thanks to the humanized behavior and the possibility to interact through the touch, the augmented plant can be seen as a companion for older adults. In this article, we present a first prototype of the system and we discuss the challenges to obtain a final product that older adults could use easily at home.

This workshop aims at discussing the rich possibilities that the body offers to experience the external world and the prospects that arise for interaction designers when these often-neglected abilities are taken into account. In particular, the workshop will focus on the rediscovery of the five senses, either alone or in a multimodal combination, and of the perceptual-motor abilities of our body. The one-day workshop is divided in two steps: first, participants will have the possibility to briefly present and discuss with the other attendees their research. Workshop candidates are requested to send a position paper, including a short biography and detailing their research interests and background. Second, participants will have the possibilities to explore and rediscover their sensorimotor abilities through several exercises and games abilities using a critical design approach. Interdisciplinary groups will be challenged to design and develop new interaction experience concepts using our natural 'tools' as prototyping tools.

We present eRS, an open-source system whose purpose is to facilitate the workflow of emotion recognition in movies, released under the MIT license. The system consists of a Django project and an AngularJS web application. It allows to easily create emotional video datasets, process the videos, extract the features and model the emotion. All data is exposed by a REST API, making it available not only to the eRS web application, but also to other applications. All visualizations are interactive and linked to the playing video, allowing researchers to easily analyze the results of their algorithms. The system currently runs on Linux and OS X. eRS can be extended, to integrate new features and algorithms needed in the different steps of emotion recognition in movies.

The goal of the system presented in this demo is to make possible for the visually and hearing impaired audience to live empathetic viewing experiences using their home theatre. In this work we suggest the incorporation of new emotion communication modalities into the standard television, to provide the targeted audience with sensations that they do not have the opportunity to enjoy because of their disability.

The accelerating progress in science with the active role of the communication media -mainly the web -make person in front of a difficult task, in finding appropriate information during a brief time.In a narrower context, many researches were created in the expertise retrieval domain, as an interesting and complicated task for the scientific community, in face of this huge amount of data scattered across the web.Benefiting from the semantic web technologies and the efforts of data structuring, in this paper we propose a novel approach of correlation based profile building, by exploiting heterogynous web sources.The aim is to generate comprehensive and validated profiles about researchers and experts in the computer science domain.

This paper presents the concept of a lamp that allows displaying and collecting user's emotional states. In particular, it displays the emotional information changing colors and facial expressions; in fact, the lamp is characterized by anthropomorphic form and behavior in order to make the interaction more natural and spontaneous. The user can interact with the lamp through tangible gestures typically used in social interactions by humans. Two different scenarios involving the use of the lamp as a companion and for computer-mediated communication are presented.

More and more objects of our everyday environment are becoming smart and connected, offering us new interaction possibilities. Tangible interaction and gestural interaction are promising communication means with these objects in this post-WIMP interaction era. Although based on different principles, they both exploit our body awareness and our skills to provide a richer and more intuitive interaction. Occasionally, when user gestures involve physical artifacts, tangible interaction and gestural interaction can blend into a new paradigm, i.e., tangible gesture interaction [5]. This workshop fosters the comparison among these different interaction paradigms and offers a unique opportunity to discuss their analogies and differences, as well as the definitions, boundaries, strengths, application domains and perspectives of tangible gesture interaction. Participants from different backgrounds are invited.

"Eyes on the road, hands on the wheel" is a crucial principle to be taken into account designing interactions for current in-vehicle interfaces. Gesture interaction is a promising modality that can be implemented following this principle in order to reduce driver distraction and increase safety. We present the results of a user elicitation for gestures performed on the surface of the steering wheel. We asked to 40 participants to elicit 6 gestures, for a total of 240 gestures. Based on the results of this experience, we derived a taxonomy of gestures performed on the steering wheel. The analysis of the results offers useful suggestions for the design of in-vehicle gestural interfaces based on this approach.

Social awareness streams limit the expressivity of emotions in computer mediated communication. In this demo, we present a system that allows sharing emotional states in a social group with multimodal ambient feedback in order to provide a more immersive and natural interaction experience.

In this paper, we present a novel opportunistic paradigm for in-vehicle gesture recognition. This paradigm allows using two or more subsystems in a synergistic manner: they can work in parallel but the lack of some of them does not compromise the functioning of the whole system. In order to segment and recognize micro-gestures performed by the user on the steering wheel, we combine a wearable approach based on the electromyography of the user's forearm muscles, with an environmental approach based on pressure sensors integrated directly on the steering wheel. We present and analyze several fusion methods and gesture segmentation strategies. A prototype has been developed and evaluated with data from nine subjects. The results prove that the proposed opportunistic system performs equal or better than each stand-alone subsystem while increasing the interaction possibilities.

In this paper, we present a research oriented open challenge focusing on multimodal gesture spotting and recognition from continuous sequences in the context of close human-computer interaction. We contextually outline the added value of the proposed challenge by presenting most recent and popular challenges and corpora available in the field. Then we present the procedures for data collection, corpus creation and the tools that have been developed for participants. Finally we introduce a novel single performance metric that has been developed to quantitatively evaluate the spotting and recognition task with multiple sensors.

In this paper we propose an approach to address the gesture segmentation issue, an important concern strongly related to the gesture recognition field. Gesture segmentation has two main goals: first, detecting when a gesture begins and ends, second, understanding whether a gesture is meant to be meaningful for the machine or is a non-command gesture (such as gesticulation). This work proposes a novel hands-free, always-available approach for the gesture segmentation and recognition in which the user can communicate directly to the system through a wearable and "intimate" interface based on electromyography signals (EMG). The system addresses the well-known "gorilla-arm" problem recognizing subtle gestures and segmenting them through motionless gestures. We report experimental results indicating that the system is able to reliably detect and recognize subtle gestures, with minimal training across users with different muscle volumes, representing a consistent gesture segmentation approach. Finally, the usability tests showed that the system is easy to use and the subjects felt quickly confident with its utilization.

In this paper we propose a novel interaction approach, based on gestures, aiming to recognize and enhance interactions between augmented human beings and augmented environments. We explain how this model, we called ARAMIS, allows interaction designers to fill the gap between real and virtual worlds augmenting the human itself. An enhanced interaction is achieved exploiting a hybrid approach. This approach is defined hybrid since it is the combination of several complementary techniques: wearable and pervasive computing paradigms, brute force, fuzzy and ML methods, virtual and real worlds, optical and non-optical sensing technologies. A framework implementing this concept has been developed. Finally, in order to validate our approach, we present a first prototype implemented according to the ARAMIS concept.

In this paper, we describe a multimodal approach for human-smart environment interaction. The input interaction is based on three modalities: deictic gestures, symbolic gestures and isolated-words. The deictic gesture is interpreted using the PTAMM (Parallel Tracking and Multiple Mapping) method exploiting a camera handheld or worn on the user arm. The PTAMM algorithm tracks in real-time the position and orientation of the hand in the environment. This information is used to point real or virtual objects, previously added to the environment, using the optical camera axis. Symbolic hand-gestures and isolated voice commands are recognized and used to interact with the pointed target. Haptic and acoustic feedbacks are provided to the user in order to improve the quality of the interaction. A complete prototype has been realized and a first usability evaluation, assessed with the help of 10 users has shown positive results.

In this paper, we describe a multimodal application, called WiiNote, facilitating multi-user photo annotation activity. The application allows up to 4 users to simultaneously annotating their pictures adding either textual or vocal comments. Users use the Wii Remote device to select the whole picture or a specific region of it to be annotated. Annotations can be either free or structured, i.e. based on a domain specific data model expressed using MPEG7 standard or RDF language for ontology.

The evolution of the web in the last decades has created the need for new requirements towards intelligent information retrieval capabilities and advanced user interfaces. Nowadays, effective retrieval and usage of multimedia resources have to deal with the issues of creating efficient indexes, developing retrieval tools and improving user oriented visualization interfaces. To that end we put forward an integrated framework named CALIMERA. The framework is based on a High-level modEL for cOnference (HELO) and aims at enhancing the information management, retrieval and visualization of recorded talks of scientific conferences. This paper presents the conference model HELO developed to perform high level annotation of scientific talk recordings, to allow granular search facilities and complex queries, and to enhance knowledge retrieval and visualization of the recordings. As a proof-of-concept a prototype has been implemented and is presented in this paper.

Tangible User Interfaces (TUIs) are emerging as a new paradigm of interaction with the digital world aiming at facilitating traditional GUI-based interaction. Interaction with TUIs relies on users' existing skills of interaction with the real world [9], thereby offering the promise of interfaces that are quicker to learn and easier to use. Recently it has been demonstrated [1] that the use of personal objects as tangible interfaces will be even more straightforward since users already have a mental model associated to the physical objects thus facilitating the comprehension and usage modalities of that objects. However TUIs are currently very challenging to build and this limits their widespread diffusion and exploitation. In order to address this issue we propose a user-oriented framework, called Memodules Framework, which allows the easy creation and management of Personal TUIs, providing end users with the ability of dynamically configuring and reconfiguring their TUIs. The framework is based on a model, called MemoML (Memodules Markup Language), which guarantees framework flexibility, extensibility and evolution over time.