VLDB 2026 Research / reviewers in the wild / expert
Tuuli Keskinen
dblp:53/11460 · also Tuuli Laivo
· DBLP profile ↗
24ranked-venue papers
5as first author
0since 2021 · last 2020
0000-0002-0170-5016ORCID · verified
Domains — the database's venue-derived domains; a paper can count in several
Human-computer interaction and ubiquitous computing · 20 · 4 first-authorGraphics, computer vision, multimedia, augmented reality and games · 7 · 1 first-authorArtificial intelligence and machine learning · 2Applied, interdisciplinary, general and emerging computing · 1 · 1 first-author
Expertise — from the expertise taxonomy: the topics of the expert's papers under the CCF categories. A weight counts papers with recency: 1 for a paper about the topic, 0.3 when the topic is its context, halved every five years.
| Human-computer interaction and pervasive computing
2 papers |
Immersive interaction · 53% Wearable and physiological sensing · 18% Usability and user experience research · 18% | |
| Computer graphics and multimedia
1 paper |
Virtual and augmented reality · 56% Visualization and visual analytics · 44% |
Topics — the 8 heaviest of 9, each with the papers that count most for it
| Topic | Weight | Papers | Last | Evidence papers |
|---|---|---|---|---|
Usability and user experience research › evaluation methodology
AR evaluation |
0.4 | 1 | 2020 | Utilizing VR and Gaze Tracking to Develop AR Solutions for Industrial Maintenance · CHI 2020 |
Immersive interaction
augmented reality |
0.4 | 1 | 2020 | Utilizing VR and Gaze Tracking to Develop AR Solutions for Industrial Maintenance · CHI 2020 |
Immersive interaction › augmented reality
augmented reality simulation |
0.4 | 1 | 2020 | Utilizing VR and Gaze Tracking to Develop AR Solutions for Industrial Maintenance · CHI 2020 |
Wearable and physiological sensing
eye tracking |
0.4 | 1 | 2020 | Utilizing VR and Gaze Tracking to Develop AR Solutions for Industrial Maintenance · CHI 2020 |
Immersive interaction
virtual reality |
0.4 | 1 | 2020 | Utilizing VR and Gaze Tracking to Develop AR Solutions for Industrial Maintenance · CHI 2020 |
Virtual and augmented reality
immersive interaction |
0.4 | 1 | 2019 | The Effect of Camera Height, Actor Behavior, and Viewer Position on the User Experience of 360° Videos · VR 2019 |
Visualization and visual analytics
usability and user experience research |
0.4 | 1 | 2019 | The Effect of Camera Height, Actor Behavior, and Viewer Position on the User Experience of 360° Videos · VR 2019 |
Virtual and augmented reality › immersive display
head-mounted display |
0.1 | 1 | 2019 | The Effect of Camera Height, Actor Behavior, and Viewer Position on the User Experience of 360° Videos · VR 2019 |
Methods — techniques the papers use, named apart from their topics
survey · 0.4iterative design · 0.4user study · 0.4exploratory research · 0.3case study · 0.3
| Year | Publication | Venue | Position |
|---|---|---|---|
| 2020 | Utilizing VR and Gaze Tracking to Develop AR Solutions for Industrial MaintenanceabstractAugmented reality (AR) presents a variety of possibilities for industrial maintenance. However, the development of real-world AR solutions has been limited due to the technological capabilities and uncertainty with respect to safety at deployment. We introduce the approach of using AR simulation in virtual reality (VR) coupled with gaze tracking to enable resource-efficient AR development. We tested in-field AR guidance and safety awareness features in an iterative development-evaluation process with experts from the elevator maintenance industry. We further conducted a survey, utilizing actual gaze data from the evaluation to elicit comments from industry experts on the usefulness of AR simulation and gaze tracking. Our results show the potential of AR within VR approach combined with gaze tracking. With this framework, AR solutions can be iteratively and safely tested without actual implementation, while gaze data provide advanced objective means to evaluate the designed AR content, documentation usage, and safety awareness. Alisa Burova, John Mäkelä, Jaakko Hakulinen, Tuuli Keskinen, Hanna Heinonen, Sanni Siltanen, Markku Turunen |
CHI | 4 |
| 2019 | Exploring Globally Inclusive Online Collaboration for Indian and Finnish SchoolchildrenabstractWe present our exploratory work on globally inclusive online collaboration for schoolchildren in India and Finland, using CityCompass, an online virtual navigational application for conversational language learning. User studies with Indian children, who collaborated with a Finnish researcher, showed several barriers towards communication and collaboration, including issues from limited access to computers and gaming to socio-cultural effects of a large power distance and face-saving. By adding a dramatized scenario, the Bollywood Method, these barriers were reduced. Next, we replicated the study with Finnish children, who collaborated with an Indian researcher. Here, previous computer and gaming experience reduced the motivation towards communication. In this paper, we present studies conducted in Finland, compare it with the Indian ones, and discuss opportunities for inclusive collaboration between Indian and Finnish children. Overall, our findings indicate that online collaboration is affected by differences in computer skills, video gaming experience, and socio-cultural communication norms. Sumita Sharma, Pekka Kallioniemi, Jaakko Hakulinen, Tuuli Keskinen, Markku Turunen |
IDC | 4 |
| 2019 | Promoting local culture and enriching airport experiences through interactive storytellingabstractExperiences in airports may shape future travel plans and contribute to tourism destination development. However, a chaotic environment and time-consuming procedural routines in airports may result in negative associations towards the host country and its culture. Despite the existence of assistive airport applications, little attention is given to facilitating travelers' engagement with cultural exploration. Alisa Burova, Chelsea Kelling, Tuuli Keskinen, Jaakko Hakulinen, Pekka Kallioniemi, Heli Väätäjä, Markku Turunen |
MUM | 3 |
| 2019 | The Effect of Camera Height, Actor Behavior, and Viewer Position on the User Experience of 360° Videosabstract360° videos can be viewed in an immersive manner with a head-mounted display (HMD). However, it is unclear how the viewing experience is affected by basic properties of 360° videos, such as how high they are recorded from, and whether there are people close to the camera. We conducted a 24-participant user study where we explored whether the viewing experience is affected by A) camera height, B) the proximity and actions of people appearing in the videos, and C) viewer position (standing/sitting). The results, surprisingly, suggest that the viewer's own height has little to no effect on the preferred camera height and the experience. The most optimal camera height situates at around 150 centimeters, which hits the comfortable height range for both sitting and standing viewers. Moreover, in some cases, people being close to the camera, or the camera being very low, has a negative effect on the experience. Our work contributes to understanding and designing immersive 360° experiences. Tuuli Keskinen, Ville Mäkelä, Pekka Kallionierni, Jaakko Hakulinen, Jussi Karhu, Kimmo Ronkainen, John Mäkelä, Markku Turunen |
VR | 1 |
| 2018 | Overcoming socio-technical challenges for cross-cultural collaborative applicationsabstractOur research focuses on facilitating access to cross-cultural collaborative applications for schoolchildren. In this paper, we present two user studies with students from an underprivileged region in Delhi. In the first study, we found that Indian students were hesitant when using the application as compared with Finnish students in a similar study, for example. We purport these challenges were social - related to face-saving and power-distance - and technical - related to lack of access to and experience with computers. In the second study, we presented a dramatized scenario before the task to address the socio-technical challenges. Our findings suggest that adding a dramatized scenario facilitated meaningful collaboration by (a) overcoming the lack of previous computer and gaming experience and (b) reducing social and cultural barriers towards communication. We believe these findings extend to cross-cultural collaboration between students with varying computer experience. Sumita Sharma, Pekka Kallioniemi, Tomi Heimonen, Jaakko Hakulinen, Markku Turunen, Tuuli Keskinen |
IDC | 6 |
| 2017 | Omnidirectional Video in Museums - Authentic, Immersive and Entertaining
Jaakko Hakulinen, Tuuli Keskinen, Ville Mäkelä, Santeri Saarinen, Markku Turunen |
ACE | 2 |
| 2017 | Utilizing Experience Goals in Design of Industrial SystemsabstractThe core idea of experience-driven design is to define the intended experience before functionality and technology. This is a radical idea for companies that have built their competences around specific technologies. Although many technology companies are willing to shift their focus towards experience-driven design, reports on real-life cases about the utilization of this design approach are rare. As part of an industry-led research program, we introduced experience-driven design to metal industry companies with experience goals as the key technique. Four design cases in three companies showed that the goals are useful in keeping the focus on user experience, but several challenges are still left for future research to tackle. This exploratory research lays ground for future research by providing initial criteria for assessing experience design tools. The results shed light on utilizing experience goals in industrial design projects and help practitioners in planning and managing the product design process with user experience in mind. Virpi Roto, Eija Kaasinen, Tomi Heimonen, Hannu Karvonen, Jussi P. P. Jokinen, Petri Mannonen, Hannu Nousu, Jaakko Hakulinen, Pertti Saariluoma, Tiina Kymäläinen, Tuuli Keskinen, Markku Turunen, Hanna Koskinen |
CHI | 12 |
| 2017 | Effect of gender on immersion in collaborative iODV applicationsabstractInteractive omnidirectional video (iODV) is a media format that allows the user to explore and interact with a 360-degree view of the recorded scenery. Recently, novel collaborative applications for presenting iODV content have emerged. Often their goal is to offer as immersive as possible experience to the users. Previous studies suggest that gender affects the §of immersion in virtual environments and other media, but there has been only little research on immersion in iODVs, and nothing in the context of collaborative interaction. In this research, we studied gender effect with participants (N=30, 15 pairs) performing a collaborative wayfinding task. Subjective data gathered with a customized immersion questionnaire showed statistically significant differences between male and female participants in Spatial Immersion and Involvement subscales. There were no statistical differences in Interaction, Realness, Physical and Auditory subscales. Several possibly affecting factors were observed during the task completion. Our results also indicate that performing interactive, collaborative tasks in iODV applications helps building a shared understanding between the users. Pekka Kallioniemi, Tuuli Keskinen, Jaakko Hakulinen, Markku Turunen, Jussi Karhu, Kimmo Ronkainen |
MUM | 2 |
| 2015 | Collaborative navigation in virtual worlds: how gender and game experience influence user behaviorabstractThere exists a large base of evidence for gender differences in human navigation. However, there is not much research on gender differences in collaborative aspects of navigation, including the interaction of individuals during collaborative wayfinding tasks in virtual environments. In light of this, we present a study of a collaborative virtual environment, Berlin Kompass. The goal of this study was to find out the main differences between genders in collaborative wayfinding. The application was evaluated in the context of foreign language learning in schools with over 200 students, where the users navigated through cityscapes while interacting verbally with each other. We collected and analyzed interaction logs, questionnaire data and audio and video recordings to gain insights into gender-related differences in wayfinding in virtual worlds. Our findings suggest that several differences that are evident in single user systems are not present when the collaborative aspect is added. Male users were more immersed during the task than females. One of the explaining factors for this might be video game experience. Genders also communicated differently -- males spoke in longer utterances whereas females had more, shorter utterances. Males referred more to relative directions and dynamic landmarks such as cars and pedestrians while navigating. Males with more video game experience also provided more positive subjective user experience feedback on the application. Pekka Kallioniemi, Tomi Heimonen, Markku Turunen, Jaakko Hakulinen, Tuuli Keskinen, Laura Pihkala-Posti, Jussi Okkonen, Roope Raisamo |
VRST | 5 |
| 2015 | Defining user experience goals to guide the design of industrial systemsabstractThe key prerequisite for experience-driven design is to define what experience to design for. User experience (UX) goals concretise the intended experience. Based on our own case studies from industrial environments and a literature study, we propose five different approaches to acquiring insight and inspiration for UX goal setting: Brand, Theory, Empathy, Technology, and Vision. Each approach brings in a different viewpoint, thus supporting the multidisciplinary character of UX. The Brand approach ensures that the UX goals are in line with the company's brand promise. The Theory approach utilises the available scientific knowledge of human behaviour. The Empathy approach focuses on knowing the actual users and stepping into their shoes. The Technology approach considers the new technologies that are being introduced and their positive or negative influence on UX. Finally, the Vision approach focuses on renewal, introducing new kinds of UXs. In the design of industrial systems, several stakeholders are involved and they should share common design goals. Using the different UX goal-setting approaches together brings in the viewpoints of different stakeholders, thus committing them to UX goal setting and emphasising UX as a strategic design decision. Eija Kaasinen, Virpi Roto, Jaakko Hakulinen, Tomi Heimonen, Jussi P. P. Jokinen, Hannu Karvonen, Tuuli Keskinen, Hanna Koskinen, Pertti Saariluoma, Helena Tokkonen, Markku Turunen |
Behav. Inf. Technol. | 7 |
| 2013 | Creating Immersive Audio and Lighting Based Physical Exercise Games for Schoolchildren
Jaakko Hakulinen, Markku Turunen, Tomi Heimonen, Tuuli Keskinen, Antti Sand, Janne Paavilainen, Jaana Parviainen, Sari Yrjänäinen, Frans Mäyrä, Jussi Okkonen, Roope Raisamo |
Advances in Computer Entertainment | 4 |
| 2013 | Designing Gesture-Based Control for Factory Automation
Tomi Heimonen, Jaakko Hakulinen, Markku Turunen, Jussi P. P. Jokinen, Tuuli Keskinen, Roope Raisamo |
INTERACT (2) | 5 |
| 2013 | Seek'N'Share: a platform for location-based collaborative mobile learningabstractWe present a location-based collaborative mobile learning platform called Seek'N'Share. It is comprised of a Web-based learning assignment editor and a mobile application for exploring and capturing multimedia content in the field. The editor enables drag-and-drop creation of learning tasks, areas and points of interest using an intuitive Web interface. Assignments are accessed with an Android application that uses location information to provide content and tasks to learners as they explore the environment. The mobile application enables the learners to record audio, video and take pictures of their environments. This supports the overall goal of putting together a presentation as the outcome of the learning activity by combining predefined, contextual information with user-generated content. The platform is currently piloted with local schools. Its novelty lies in its flexible support for creating location-based learning activities for unconstrained environments, and the possibility for the learners to collaboratively document their learning outcomes in situ. Tomi Heimonen, Markku Turunen, Sanna Kangas, Tamás Pallos, Pasi Pekkala, Santeri Saarinen, Katariina Tiitinen, Tuuli Keskinen, Matti Luhtala, Olli Koskinen, Jussi Okkonen, Roope Raisamo |
MUM | 8 |
| 2013 | Evaluating landmark attraction model in collaborative wayfinding in virtual learning environmentsabstractIn Virtual Learning Environments efficient navigation is a major issue, especially when it is used as a component in the learning process. This paper addresses the challenges in creating meaningful navigation routes from language learning perspective. The work is grounded on findings from a specific case on German language learning, wherein two remotely located users communicated in a wayfinding guidance scenario. The users navigated through 360-degree virtual panoramic images using body gestures and could receive communication help via spoken hints by pointing at objects in the scenery. An important design consideration is how to choose these objects, as they have both navigational importance and pedagogical significance in terms of learning the desired language. Wayfinding interactions from 21 participants were compared to the values provided by a landmark attraction model applied on the landmarks along the routes. The results show that there was a clear connection between prominence of landmarks and time spent on each panorama. This indicates that together with pedagogical planning, the model can aid in selecting the interactive content for language learning applications in virtual environments. Pekka Kallioniemi, Jaakko Hakulinen, Tuuli Keskinen, Markku Turunen, Tomi Heimonen, Laura Pihkala-Posti, Mikael Uusi-Mäkelä, Pentti Hietala, Jussi Okkonen, Roope Raisamo |
MUM | 3 |
| 2013 | Evaluating the experiential user experience of public display applications in the wildabstractStudying pervasive systems in the wild has recently gained significant interest. However, few methods exist that focus on the subjective of user experience of such systems rather than objective metrics, like performance and task success. Especially multimodal interaction in this context poses challenges to understanding how different input and output methods affect the users' experience. We present a new method for evaluating the experiential user experience of interactive systems. It combines two existing approaches from different fields: a questionnaire-based evaluation method called SUXES, intended for evaluating user expectations and experiences, and a theoretical experience framework, Experience Pyramid, originally developed for analyzing and improving experiential tourism products. The new method was used in two field studies of multimodal public display applications. Our findings show that the method is a practical approach for user experience evaluation in the wild, especially in the case of pervasive applications that aim to provide novel experiences rather than facilitate task-oriented information access. Tuuli Keskinen, Jaakko Hakulinen, Tomi Heimonen, Markku Turunen, Sumita Sharma, Toni Miettinen, Matti Luhtala |
MUM | 1 |
| 2013 | Mobile dictation for healthcare professionalsabstractWe demonstrate a mobile dictation application utilizing automatic speech recognition for healthcare professionals. Development was done in close collaboration between human-technology interaction and nursing science researchers and professionals working in the area. Our work was motivated by the need for improvements in getting spoken patient information to the next treatment steps without additional steps. In addition, we wanted to enable truly mobile spoken information entry, i.e., dictation can take place on the spot. In order to study the applicability we conducted a small-scale Wizard-of-Oz evaluation in a real hospital environment with real nurses. Our main focus was to gather subjective expectations and experiences from the actual nurses themselves. The results show true potential for our mobile dictation application and its further development. Tuuli Keskinen, Aleksi Melto, Jaakko Hakulinen, Markku Turunen, Santeri Saarinen, Tamás Pallos, Pekka Kallioniemi, Riitta Danielsson-Ojala, Sanna Salanterä |
MUM | 1 |
| 2012 | SymbolChat: Picture-Based Communication Platform for Users with Intellectual Disabilities
Tuuli Keskinen, Tomi Heimonen, Markku Turunen, Juha-Pekka Rajaniemi, Sami Kauppinen |
ICCHP (2) | 1 |
| 2012 | SymbolChat: A flexible picture-based communication platform for users with intellectual disabilitiesabstractPersons with intellectual disabilities benefit from participating in the modern information society, especially the World Wide Web, social media and Internet-mediated communication services. Although several computer-based prototypes and commercial systems have been introduced for accessible in-person communication, currently few applications and services exist to support synchronous remote communication for this user group. We introduce SymbolChat, a software platform that supports the creation of multimodal communication applications utilizing picture-based instant messaging. End users and their support personnel can customize the input and output features of the application based on their individual needs and abilities. The interaction is based on touchscreen input and speech output using speech synthesis technology. The SymbolChat platform was developed together with the prospective end users and practitioners in the field of special needs care. We evaluated the prototype application in a field study with nine users with varying degrees of intellectual and other disabilities. The results clearly indicate that the participants were able to express themselves in spontaneous communication using a large-scale picture-based vocabulary (around 2000 symbols) even without prior training in the use of symbols. This finding was supported in the constructive feedback gathered from professionals working in the area. We also successfully applied methodology from other settings, such as child-computer interaction to evaluate interaction in this challenging context. Overall, the results show that social inclusion for people with intellectual disabilities can be improved with customizable communication tools. The implemented communication platform forms a solid basis for further improvements and new communication services. In addition, we found that users with motor impairments would greatly benefit from alternative input and output methods for symbol browsing and selection. Tuuli Keskinen, Tomi Heimonen, Markku Turunen, Juha-Pekka Rajaniemi, Sami Kauppinen |
Interact. Comput. | 1 |
| 2009 | Multimodal Media Center Interface Based on Speech, Gestures and Haptic Feedback
Markku Turunen, Jaakko Hakulinen, Juho Hella, Juha-Pekka Rajaniemi, Aleksi Melto, Erno Mäkinen, Jussi Rantala, Tomi Heimonen, Tuuli Keskinen, Hannu Soronen, Mervi Hansen, Pellervo Valkama, Toni Miettinen, Roope Raisamo |
INTERACT (2) | 9 |
| 2009 | Multimodal Interaction with Speech, Gestures and Haptic Feedback in a Media Center Application
Markku Turunen, Jaakko Hakulinen, Juho Hella, Juha-Pekka Rajaniemi, Aleksi Melto, Erno Mäkinen, Jussi Rantala, Tomi Heimonen, Tuuli Keskinen, Hannu Soronen, Mervi Hansen, Pellervo Valkama, Toni Miettinen, Roope Raisamo |
INTERACT (2) | 9 |
| 2009 | SUXES - user experience evaluation method for spoken and multimodal interactionabstractMuch work remains to be done with subjective evaluations of speech-based and multimodal systems. In particular, user experience is still hard to evaluate. SUXES is an evaluation method for collecting subjective metrics with user experiments. It captures both user expectations and user experiences, making it possible to analyze the state of the application and its interaction methods, and compare results. We present the SUXES method with examples of user experiments with different applications and modalities. Index Terms: evaluation, subjective metrics, user experience Markku Turunen, Jaakko Hakulinen, Aleksi Melto, Tomi Heimonen, Tuuli Keskinen, Juho Hella |
INTERSPEECH | 5 |
| 2009 | Speech-based and multimodal media center for different user groups
Markku Turunen, Jaakko Hakulinen, Aleksi Melto, Juho Hella, Juha-Pekka Rajaniemi, Erno Mäkinen, Jussi Rantala, Tomi Heimonen, Tuuli Keskinen, Hannu Soronen, Mervi Hansen, Pellervo Valkama, Toni Miettinen, Roope Raisamo |
INTERSPEECH | 9 |
| 2009 | Visual Perception of Parallel Coordinate VisualizationsabstractParallel coordinates is a visualization technique that provides an unbiased representation of high-dimensional data. The parallel configuration of axes treats data dimensions uniformly and is well suited for exploratory visualization. However, first-time users of parallel coordinate visualizations can find the representation confusing and difficult to understand.We used eye tracking to study how parallel coordinate visualizations are perceived, and compared the results to the optimal visual scan path required to complete the tasks. The results indicate that even first-time users quickly learn how to use parallel coordinate visualizations, pay attention to the correct task-specific areas in the visualization, and become rapidly proficient with it. Harri Siirtola, Tuuli Keskinen, Tomi Heimonen, Kari-Jouko Räihä |
IV | 2 |
| 2009 | User expectations and user experience with different modalities in a mobile phone controlled home entertainment systemabstractHome environment is an exciting application domain for multimodal mobile interfaces. Instead of multiple remote controls, personal mobile devices could be used to operate home entertainment systems. This paper reports a subjective evaluation of multimodal inputs and outputs for controlling a home media center using a mobile phone. A within-subject evaluation with 26 participants revealed significant differences on user expectations on and experiences with different modalities. Speech input was received extremely well, even surpassing expectations in some cases, while gestures and haptic feedback were almost failing to meet the lowest expectations. The results can be applied for designing similar multimodal applications in home environments. Markku Turunen, Aleksi Melto, Juho Hella, Tomi Heimonen, Jaakko Hakulinen, Erno Mäkinen, Tuuli Keskinen, Hannu Soronen |
Mobile HCI | 7 |