Virtual reality headsets enabling immersive experiences have been used for a long time in various public spaces such as universities, vocational education, and libraries. There is a plethora of different content available for VR headsets that can support education or entertainment, such as games and movies that enrich users' lives. However, there is little literature on the use of VR games by older adults. In addition, the understanding of how older adults experience virtual worlds and games is still limited. It is believed that they experience barriers with gaming and may consider it a waste of time. Older adults, however, can become active gamers if they are offered adequate support measures and when age-related factors are taken into account in the design of the games. Video games can enrich the lives of older adults and aid in managing aging challenges. Thus, it is important to study older adults' playful experiences with VR games. This study examines older adults' playful experiences of VR games and how they experience a VR gaming session in general. Through field observations collected from 33 participants during seven gaming sessions, the results point to several issues related to enjoyable, meaningful, and immersive game experiences: Confusion, fear, thrill, excitement, interest, curiosity, and enjoyment were the most common playful experiences in these VR games. The study revealed that in some cases, older players experienced fear of falling due to the content of the VR world. Most of the participants in the study were excited and some even wanted to continue using the VR glasses after the field experiment ended. This highlights the potential of VR gaming, necessitating further research into the playful experiences of VR gaming with older adults.

Augmented reality (AR) presents a variety of possibilities for industrial maintenance. However, the development of real-world AR solutions has been limited due to the technological capabilities and uncertainty with respect to safety at deployment. We introduce the approach of using AR simulation in virtual reality (VR) coupled with gaze tracking to enable resource-efficient AR development. We tested in-field AR guidance and safety awareness features in an iterative development-evaluation process with experts from the elevator maintenance industry. We further conducted a survey, utilizing actual gaze data from the evaluation to elicit comments from industry experts on the usefulness of AR simulation and gaze tracking. Our results show the potential of AR within VR approach combined with gaze tracking. With this framework, AR solutions can be iteratively and safely tested without actual implementation, while gaze data provide advanced objective means to evaluate the designed AR content, documentation usage, and safety awareness.

We present our exploratory work on globally inclusive online collaboration for schoolchildren in India and Finland, using CityCompass, an online virtual navigational application for conversational language learning. User studies with Indian children, who collaborated with a Finnish researcher, showed several barriers towards communication and collaboration, including issues from limited access to computers and gaming to socio-cultural effects of a large power distance and face-saving. By adding a dramatized scenario, the Bollywood Method, these barriers were reduced. Next, we replicated the study with Finnish children, who collaborated with an Indian researcher. Here, previous computer and gaming experience reduced the motivation towards communication. In this paper, we present studies conducted in Finland, compare it with the Indian ones, and discuss opportunities for inclusive collaboration between Indian and Finnish children. Overall, our findings indicate that online collaboration is affected by differences in computer skills, video gaming experience, and socio-cultural communication norms.

Experiences in airports may shape future travel plans and contribute to tourism destination development. However, a chaotic environment and time-consuming procedural routines in airports may result in negative associations towards the host country and its culture. Despite the existence of assistive airport applications, little attention is given to facilitating travelers' engagement with cultural exploration.

360° videos can be viewed in an immersive manner with a head-mounted display (HMD). However, it is unclear how the viewing experience is affected by basic properties of 360° videos, such as how high they are recorded from, and whether there are people close to the camera. We conducted a 24-participant user study where we explored whether the viewing experience is affected by A) camera height, B) the proximity and actions of people appearing in the videos, and C) viewer position (standing/sitting). The results, surprisingly, suggest that the viewer's own height has little to no effect on the preferred camera height and the experience. The most optimal camera height situates at around 150 centimeters, which hits the comfortable height range for both sitting and standing viewers. Moreover, in some cases, people being close to the camera, or the camera being very low, has a negative effect on the experience. Our work contributes to understanding and designing immersive 360° experiences.

Our research focuses on facilitating access to cross-cultural collaborative applications for schoolchildren. In this paper, we present two user studies with students from an underprivileged region in Delhi. In the first study, we found that Indian students were hesitant when using the application as compared with Finnish students in a similar study, for example. We purport these challenges were social - related to face-saving and power-distance - and technical - related to lack of access to and experience with computers. In the second study, we presented a dramatized scenario before the task to address the socio-technical challenges. Our findings suggest that adding a dramatized scenario facilitated meaningful collaboration by (a) overcoming the lack of previous computer and gaming experience and (b) reducing social and cultural barriers towards communication. We believe these findings extend to cross-cultural collaboration between students with varying computer experience.

In this paper, we build a case for incorporating socio-technical aspirations of different stakeholders, e.g. parents, care-givers, and therapists, to motivate technology acceptance and adoption for children with autism. We base this on findings from two studies at a special school in New Delhi. First, with six children with autism, their parents and therapists we explored whether fitness bands motivate children with autism in India to increase their physical activity. Second, with five parents and specialists at the same school, we conducted interviews to understand their expectations from and current usage of technology. Previous work defines a culture-based framework for assistive technology design with three dimensions: lifestyle, socio-technical infrastructure, and monetary and informational resources. To this framework we propose a fourth dimension of socio-technical aspirations. We discuss the implications of the proposed fourth dimension to the existing framework.

We present Pocket Transfers: interaction techniques that allow users to transfer content from situated displays to a personal mobile device while keeping the device in a pocket or bag. Existing content transfer solutions require direct manipulation of the mobile device, making inter-action slower and less flexible. Our introduced tech-niques employ touch, mid-air gestures, gaze, and a mul-timodal combination of gaze and mid-air gestures. We evaluated the techniques in a novel user study (N=20), where we considered dynamic scenarios where the user approaches the display, completes the task, and leaves. We show that all pocket transfer techniques are fast and seen as highly convenient. Mid-air gestures are the most efficient touchless method for transferring a single item, while the multimodal method is the fastest touchless method when multiple items are transferred. We provide guidelines to help researchers and practitioners choose the most suitable content transfer techniques for their systems.

This article presents a scalable, semi-automated process for studying the usage of public displays. The process consists of gathering anonymous interaction and skeletal data of passersby during public display deployment and programmatically analyzing the data. This article demonstrates the use of the process with the analysis of the Information Wall, a gesture-controlled public information display. Information Wall was deployed in a university campus for one year and collected an extensive data set of more than 100 000 passersby. The main benefits of the process include (1) gathering of large data sets without considerable use of resources, (2) fast, semi-automated data analysis, and (3) applicability to studying the effects of long-term public display deployments. In analyzing the usage and passersby data of the Information Wall in our validation study, the main findings uncovered using the method were (i) most users were first-time users exploring the system, and not many returned to use the system again, and (ii) many users were accompanied by passive users who observed interaction from further away, which could suggest a case of multi-user interaction blindness. In the past, logged data has mainly been used as a supporting method for in situ observations and interviews, and its use has required a considerable amount of manual work. In this article, we argue that logged data analysis can be automated to complement other methods, particularly in the evaluation of long-term deployments.

Public display deployments are often subjected to various surprising and unwanted effects. These effects are frequently due to external factors properties and phenomena that are unrelated to the deployment. Therefore, we conducted a literature review within the public display domain to investigate the causes behind the reported issues. This work presents a taxonomy of external factors affecting deployments, consisting of six categories: weather, events, surroundings, space, inhabitants, and vandalism. Apart from a few positive examples, we predominantly found negative effects arising from these factors. We then identified four ways of addressing the effects: ignoring, adapting, solving, and embracing. Of these, ignoring and adapting are substantially more frequent responses than solving and embracing emphasizing the need for researchers to adapt. We present real-world examples and insights on how researchers and practitioners can address the effects to better manage their deployments.

The core idea of experience-driven design is to define the intended experience before functionality and technology. This is a radical idea for companies that have built their competences around specific technologies. Although many technology companies are willing to shift their focus towards experience-driven design, reports on real-life cases about the utilization of this design approach are rare. As part of an industry-led research program, we introduced experience-driven design to metal industry companies with experience goals as the key technique. Four design cases in three companies showed that the goals are useful in keeping the focus on user experience, but several challenges are still left for future research to tackle. This exploratory research lays ground for future research by providing initial criteria for assessing experience design tools. The results shed light on utilizing experience goals in industrial design projects and help practitioners in planning and managing the product design process with user experience in mind.

Analyzing data from complex production systems and processes can be used in improving existing products, processes, and services, and innovating novel offerings. We report the findings from a six-month case study with a company developing flexible manufacturing systems. During a collaborative development process of a data analytics and visualization tool, our goal was to identify potential metrics, business opportunities, and challenges when utilizing logged data of end-users' human-machine interactions in development activities. Our key contributions include a characterization of the potential usage data metrics to be logged and visualized, identification of opportunities this data entails for business, and discussion about the challenges related to usage data logging in the studied context. Finally, we propose topics that should be considered in the organization before investing in usage data logging in the context of flexible manufacturing systems.

Interactive omnidirectional video (iODV) is a media format that allows the user to explore and interact with a 360-degree view of the recorded scenery. Recently, novel collaborative applications for presenting iODV content have emerged. Often their goal is to offer as immersive as possible experience to the users. Previous studies suggest that gender affects the §of immersion in virtual environments and other media, but there has been only little research on immersion in iODVs, and nothing in the context of collaborative interaction. In this research, we studied gender effect with participants (N=30, 15 pairs) performing a collaborative wayfinding task. Subjective data gathered with a customized immersion questionnaire showed statistically significant differences between male and female participants in Spatial Immersion and Involvement subscales. There were no statistical differences in Interaction, Realness, Physical and Auditory subscales. Several possibly affecting factors were observed during the task completion. Our results also indicate that performing interactive, collaborative tasks in iODV applications helps building a shared understanding between the users.

CityCompass promotes collaborative conversational language learning by means of way-findings tasks situated in panoramic views of a city. Two remotely located participants have to navigate these 360 city panoramas as a tourist or a guide, to arrive at a preassigned destination. The application supports three modes of interaction and immersion: traditional mouse and keyboard interaction, embodied interaction using the Microsoft Kinect sensor and a virtual reality supporting 360 degree video panoramas on a mobile phone placed inside virtual reality googles (Samsung Gear VR).

Gesture-based interaction provides a multitude of benefits to individuals with disabilities, for example, enhancing social, motor and cognitive skills. However, applications that encourage self-efficacy by promoting a life-skill through simulations of real world scenarios are largely missing. We explore the benefits of using a gesture-based application for individuals with developmental disabilities. The context is a special school in New Delhi, Nai Disha, where we designed and developed an application, Kirana, that integrates arithmetic and social interaction to teach purchasing of items from a local grocery store. In our study, 18 participants with developmental disabilities, previously unable to visit a grocery store, used Kirana for three weeks. Our results indicate that gesture-based applications can teach a life skill and enable self-efficacy for individuals with developmental disabilities by breaking down complex tasks that require social, mathematical and decision-making skills.

Kirana is a gesture-based application that simulates purchasing experience at a local Indian grocery store. It provides individuals with developmental difficulties a safe and controlled environment to explore the processes of purchasing an item: namely, deciding items to buy (decision making), calculating costs and balance (mathematical skills), and interactions via pointing (social interaction). Previous research has established that gesture-based interaction has the potential to enhance social, motor and cognitive skills for individuals with developmental disabilities. Currently, translating learnings from gesture-based virtual applications to real world scenarios is underexplored. Kirana is an approach to simulating practical real interactions, by breaking down complex tasks that require social, mathematical and decision-making skills, and encouraging self-efficacy. Skills learnt by purchasing items with Kirana, are potentially transferable to a real world store.

There exists mounting evidence in favor of computer supported autism interventions at the individual level. However, the potential benefits of using computer supported collaboration to encourage social interactions between individuals with autism and typically developed individuals are underexplored, particularly in developing regions. We present an exploratory study of a collaborative gesture-based application, Balloons. The application encourages joint attention, which is defined as the shared attention between two individuals towards the same object. Using mixed methods, we evaluated Balloons for three weeks in New Delhi with ten medium-low functioning autistic children. Our findings suggest that employing CSC interventions for children with autism in India provide (a) observable improvements in social interaction with typically developed peers, (b) the opportunity to customize and individualize intervention to cater to a large spectrum of children and (c) the potential opportunity of reducing fears of certain objects.

There exists a large base of evidence for gender differences in human navigation. However, there is not much research on gender differences in collaborative aspects of navigation, including the interaction of individuals during collaborative wayfinding tasks in virtual environments. In light of this, we present a study of a collaborative virtual environment, Berlin Kompass. The goal of this study was to find out the main differences between genders in collaborative wayfinding. The application was evaluated in the context of foreign language learning in schools with over 200 students, where the users navigated through cityscapes while interacting verbally with each other. We collected and analyzed interaction logs, questionnaire data and audio and video recordings to gain insights into gender-related differences in wayfinding in virtual worlds. Our findings suggest that several differences that are evident in single user systems are not present when the collaborative aspect is added. Male users were more immersed during the task than females. One of the explaining factors for this might be video game experience. Genders also communicated differently -- males spoke in longer utterances whereas females had more, shorter utterances. Males referred more to relative directions and dynamic landmarks such as cars and pedestrians while navigating. Males with more video game experience also provided more positive subjective user experience feedback on the application.

The key prerequisite for experience-driven design is to define what experience to design for. User experience (UX) goals concretise the intended experience. Based on our own case studies from industrial environments and a literature study, we propose five different approaches to acquiring insight and inspiration for UX goal setting: Brand, Theory, Empathy, Technology, and Vision. Each approach brings in a different viewpoint, thus supporting the multidisciplinary character of UX. The Brand approach ensures that the UX goals are in line with the company's brand promise. The Theory approach utilises the available scientific knowledge of human behaviour. The Empathy approach focuses on knowing the actual users and stepping into their shoes. The Technology approach considers the new technologies that are being introduced and their positive or negative influence on UX. Finally, the Vision approach focuses on renewal, introducing new kinds of UXs. In the design of industrial systems, several stakeholders are involved and they should share common design goals. Using the different UX goal-setting approaches together brings in the viewpoints of different stakeholders, thus committing them to UX goal setting and emphasising UX as a strategic design decision.

Public displays that allow users to interact with them through mid-air gestures are still relatively rare, as many applications rely on touch-based interaction. This paper introduces Information Wall, a gesture-controlled public information display that provides multi-user access to contextually relevant local information using remote pointing and mid-air gestures. The application has been studied in two settings: a lab-based user study and several short-term deployments. Based on our results, we present practical guidelines for gesture-controlled public display design.

The SiMPE workshop series started in 2006 with the goal of enabling speech processing on mobile and embedded devices. The SiMPE 2012 workshop extended the notion of audio to non-speech "Sounds" and thus the expansion became "Speech and Sound". SiMPE 2010 and 2011 brought together researchers from the speech and the HCI communities. Speech User interaction in cars was a focus area in 2009. Multimodality got more attention in SiMPE 2008. In SiMPE 2007, the focus was on developing regions.

We present a location-based collaborative mobile learning platform called Seek'N'Share. It is comprised of a Web-based learning assignment editor and a mobile application for exploring and capturing multimedia content in the field. The editor enables drag-and-drop creation of learning tasks, areas and points of interest using an intuitive Web interface. Assignments are accessed with an Android application that uses location information to provide content and tasks to learners as they explore the environment. The mobile application enables the learners to record audio, video and take pictures of their environments. This supports the overall goal of putting together a presentation as the outcome of the learning activity by combining predefined, contextual information with user-generated content. The platform is currently piloted with local schools. Its novelty lies in its flexible support for creating location-based learning activities for unconstrained environments, and the possibility for the learners to collaboratively document their learning outcomes in situ.

In Virtual Learning Environments efficient navigation is a major issue, especially when it is used as a component in the learning process. This paper addresses the challenges in creating meaningful navigation routes from language learning perspective. The work is grounded on findings from a specific case on German language learning, wherein two remotely located users communicated in a wayfinding guidance scenario. The users navigated through 360-degree virtual panoramic images using body gestures and could receive communication help via spoken hints by pointing at objects in the scenery. An important design consideration is how to choose these objects, as they have both navigational importance and pedagogical significance in terms of learning the desired language. Wayfinding interactions from 21 participants were compared to the values provided by a landmark attraction model applied on the landmarks along the routes. The results show that there was a clear connection between prominence of landmarks and time spent on each panorama. This indicates that together with pedagogical planning, the model can aid in selecting the interactive content for language learning applications in virtual environments.

Studying pervasive systems in the wild has recently gained significant interest. However, few methods exist that focus on the subjective of user experience of such systems rather than objective metrics, like performance and task success. Especially multimodal interaction in this context poses challenges to understanding how different input and output methods affect the users' experience. We present a new method for evaluating the experiential user experience of interactive systems. It combines two existing approaches from different fields: a questionnaire-based evaluation method called SUXES, intended for evaluating user expectations and experiences, and a theoretical experience framework, Experience Pyramid, originally developed for analyzing and improving experiential tourism products. The new method was used in two field studies of multimodal public display applications. Our findings show that the method is a practical approach for user experience evaluation in the wild, especially in the case of pervasive applications that aim to provide novel experiences rather than facilitate task-oriented information access.

We demonstrate a mobile dictation application utilizing automatic speech recognition for healthcare professionals. Development was done in close collaboration between human-technology interaction and nursing science researchers and professionals working in the area. Our work was motivated by the need for improvements in getting spoken patient information to the next treatment steps without additional steps. In addition, we wanted to enable truly mobile spoken information entry, i.e., dictation can take place on the spot. In order to study the applicability we conducted a small-scale Wizard-of-Oz evaluation in a real hospital environment with real nurses. Our main focus was to gather subjective expectations and experiences from the actual nurses themselves. The results show true potential for our mobile dictation application and its further development.

We introduce a model for landmark highlighting for pedestrian route guidance services for mobile devices. The model determines which landmarks are the most attractive based on their properties in the current context of user's orientation and the location on the route and highlights these landmarks on the mobile map. The attractiveness of a landmark is based on its visual, structural and semantic properties which are used for calculating the total attractiveness of a single landmark.

Persons with intellectual disabilities benefit from participating in the modern information society, especially the World Wide Web, social media and Internet-mediated communication services. Although several computer-based prototypes and commercial systems have been introduced for accessible in-person communication, currently few applications and services exist to support synchronous remote communication for this user group. We introduce SymbolChat, a software platform that supports the creation of multimodal communication applications utilizing picture-based instant messaging. End users and their support personnel can customize the input and output features of the application based on their individual needs and abilities. The interaction is based on touchscreen input and speech output using speech synthesis technology. The SymbolChat platform was developed together with the prospective end users and practitioners in the field of special needs care. We evaluated the prototype application in a field study with nine users with varying degrees of intellectual and other disabilities. The results clearly indicate that the participants were able to express themselves in spontaneous communication using a large-scale picture-based vocabulary (around 2000 symbols) even without prior training in the use of symbols. This finding was supported in the constructive feedback gathered from professionals working in the area. We also successfully applied methodology from other settings, such as child-computer interaction to evaluate interaction in this challenging context. Overall, the results show that social inclusion for people with intellectual disabilities can be improved with customizable communication tools. The implemented communication platform forms a solid basis for further improvements and new communication services. In addition, we found that users with motor impairments would greatly benefit from alternative input and output methods for symbol browsing and selection.

"In all the wonderful worlds that writing opens, the spoken word still resides and lives. Written texts all have to be related somehow, directly or indirectly, to the world of sound, the natural habitat of language, to yield their meanings."

With the proliferation of pervasive devices and the increase in their processing capabilities, client-side speech processing has been emerging as a viable alternative. The SiMPE workshop series started in 2006 [5] with the goal of enabling speech processing on mobile and embedded devices to meet the challenges of pervasive environments (such as noise) and leveraging the context they offer (such as location). SiMPE 2010, the latest in the series brought together, very successfully, researchers from the speech and the HCI communities. We believe this is the beginning.

We demonstrate a multimodal, multi-user, and multi-device conference program guide for conference participants. Its functionality includes access to the conference program with additional multimedia content, voting, feedback, and communication with the other participants. People can interact with the system in a multimodal way using spoken language, gestures and haptic feedback with mobile phones and shared public displays.

Awareness of shared work activities is important for fostering professional relationships. This paper introduces visualization techniques for a meeting support system based on automatic speech recognition and multisensory meeting information capture, including voice levels and participant comments. The system has been deployed in live meetings over the past 5 months and we report meeting participants’ expectations and findings from a field study. Initial findings suggest that the proposed approach can facilitate awareness without the need for detailed meeting transcriptions and associated audio recordings, however the quality of speech recognition results and their effective visualization remain critical factors for increased adoption.

With the proliferation of pervasive devices and the increase in their processing capabilities, client-side speech processing has been emerging as a viable alternative. The SiMPE workshop series started in 2006 [5] with the goal of enabling speech processing on mobile and embedded devices to meet the challenges of pervasive environments (such as noise) and leveraging the context they offer (such as location).

Much work remains to be done with subjective evaluations of speech-based and multimodal systems. In particular, user experience is still hard to evaluate. SUXES is an evaluation method for collecting subjective metrics with user experiments. It captures both user expectations and user experiences, making it possible to analyze the state of the application and its interaction methods, and compare results. We present the SUXES method with examples of user experiments with different applications and modalities. Index Terms: evaluation, subjective metrics, user experience

Home environment is an exciting application domain for multimodal mobile interfaces. Instead of multiple remote controls, personal mobile devices could be used to operate home entertainment systems. This paper reports a subjective evaluation of multimodal inputs and outputs for controlling a home media center using a mobile phone. A within-subject evaluation with 26 participants revealed significant differences on user expectations on and experiences with different modalities. Speech input was received extremely well, even surpassing expectations in some cases, while gestures and haptic feedback were almost failing to meet the lowest expectations. The results can be applied for designing similar multimodal applications in home environments.

We studied with telephone survey the opinions and ideas of 1004 Finns concerning domestic technology and controlling it with voice commands. There is distrust towards commanding your home environment with voice; the users doubt especially its functionality. Voice commands are regarded as unpleasant. The most positive conceived aspects of voice commands are the speed of control and the ability to free your hands for something else. Voice feedback, however, is really appreciated. Especially it is welcomed instead of alarm beeps and blinking lights. It is possible that users need first to get accustomed that a device speaks to them. Index Terms: smart home, speech interface, voice command, consumer survey

We present a physical multimodal conversational Companion in the area of health and fitness. Conversational spoken dialogues using physical agents provide a potential interface for applications which are aimed at motivating and supporting users. Open source software called jNabServer, which enables spoken and multimodal interaction with Nabaztag/tag wireless rabbits, is presented together with other software architecture solutions applied in the development of the Companion. We also present how the Companion manages interaction with the combination of a dialogue manager and a cognitive model. Index Terms: spoken dialogue systems, multimodality, software architectures, physical agent interfaces

Abstract Mobile devices have become ubiquitous and reasonably po-werful and well connected. However, their physical size limits possibilities of interaction, especially document creation. Dictation in mobile setting provides one solution, but limited processing power requires that the actual speech recognition is distributed to a server computer. We present MobiDic, a distributed mobile dictation application. Its user interface solutions solve problems that arise from the technical limita-tion of mobile devices and mobile user context. Index Terms : mobile applications, dictation, audio editing 1. Introduction Dictation is one of the few real success stories of speech rec-ognition technology, and speech in general can be very effi-cient data entry method [1], although there are several chal-lenges for its use [2]. Traditional dictation applications are mostly used by specific professions, e.g., doctors and lawyers, in acoustically safe environments and on computationally adequate hardware. However, people are more and more on the move most of their day, and have immediate access only to less capable hardware, both computationally and interac-tion-wise. Furthermore, their schedule is often fragmented, so they do not have time for long-lasting dictation sessions to dictate in the traditional sense. People perceive mobile dicta-tion to be very useful, but there are also great demands for its accessibility and usability [3][4]. Thus, mobile solutions al-lowing incremental dictation and notetaking are necessary to meet the current demands. However, mobile platforms, and standard mobile phones in particular, are problematic for dictation applications be-cause of lack of processing power. There are only initial pro-totypes of embedded large-vocabulary speech recognition [5]. Most advanced dictation systems need more hardware capa-bilities than is available in current and near-future mobile phones. Because of these reasons we have focused on distri-buted solution, where the dictation and editing can be done with a mobile device but ASR takes place in a server ma-chine. Furthermore, the server stores user audio recordings and recognized texts, and maintains acoustic models and dis-tributes generated document, e.g., via email. The limited interaction capabilities offered by mobile de-vices form another fundamental issue. In particular, small screens and keypads make it hard to design efficient docu-ment editing interfaces. Most of the desktop applications are assuming standard sized screens and multimodal interaction, in particular when error correction is considered [6]. Further-more, most of them are targeted for single session, real-time interaction, not asynchronous and fragmented interaction as in mobile dictation applications. There are some commercial asynchronous mobile dicta-tion solutions available. Typically, the user dictates while on move, and the recognition takes place afterwards. In this case, the process is not interactive, and the user is not able to re-view, edit and distribute the results in the mobile settings. We present MobiDic, an asynchronous dictation and note-taking application for mobile phones. In this paper, we de-scribe the MobiDic application, its functionality and architec-ture, and then discuss the user interface solutions for incre-mental dictation, audio editing, recognition results editing, and user adaptation. Figure 1.

We present initial results from an evaluation of a multimodal interface for a mobile route guidance application providing public transport information in Finland. The application includes a range of input and output modalities, such as speech input and output, a fisheye GUI, and contextual predictive text input. The application has been in public pilot use for almost a year. In this paper we present results from a formal user evaluation, focusing on predictive text input and speech input and output. Speech input outperformed other input methods, even with high error rates and slow response times. However, the domain specific predictive text input was the preferred method, surpassing users' expectations.

In the past, voice-based applications have been accessed using unintelligent telephone devices through Voice Browsers that reside on the server. The proliferation of pervasive devices and the increase in their processing capabilities, clientside speech processing has been emerging as a viable alternative. In SiMPE 2008, the third in the series, we will continue to explore the various possibilities and issues that arise while enabling speech processing on resource-constrained, possibly mobile devices.

We present a design of a rich multimodal interface for mobile route guidance. The application provides public transport information in Finland, including support for pedestrian guidance when the user is changing between the means of transportation. The range of input and output modalities include speech synthesis, speech recognition, a fisheye GUI, haptics, contextual text input, physical browsing, physical gestures, non-speech audio, and global positioning information. Together, these modalities provide an interface that is accessible for a wide range of users including persons with various levels of visual impairment. In this paper we describe the functional aspects and the design of the interface of our publicly available prototype system. Index Terms: speech interfaces, multimodal interfaces, mobile applications, accessibility

We present findings from the long-term study of a speech-based bus timetable system. After the deployment of the prototype system we have collected data from real usage for 30 months. In addition, we have conducted usability tests to get subjective ratings of the pilot system. The comparison of these evaluations shows that the results obtained with usability tests differ significantly from those gained from the real usage, and the data of the initial use differs significantly from the data collected after that. For example, the differences in help requests, interruptions, speech recognition rejections, silence timeouts, and repeat requests are highly significant, and in some cases, such as explicit quit requests, enormous (65 % versus 3%). Index Terms: spoken dialog systems, evaluation, pilot studies 1.

Graphical elements have been found very useful when spoken dialogue systems are developed and demonstrated. However, most of the spoken dialogue systems are designed for speechonly interaction and are very hard to extent to contain graphical elements. We introduce a general model to visualize speech interfaces. Based on the model we present an implemented visualization framework, and several example visualizations for demonstrations, debugging, and interactive tutoring of speech applications. 1.

Mobile devices, such as smartphones and personal digital assistants, can be used to implement efficient speech-based and multimodal interfaces. Most of the systems are serverbased, and there is a need to distribute the dialogue management tasks between the terminal devices and the server. Since the technologies are not mature and the platforms are constantly changing, approaches that support generic and reusable components are needed. We describe a model where the tasks of the dialogue management are divided into two parts. The server handles the overall coordination of the dialogue by generating dialogue task descriptions. Smart terminal devices realize the low-level decisions by executing individual dialogue tasks. A prototype of multimodal dialogue application using the model is presented. It has two different user interface realizations using standard telephony equipment and smartphones. 1.

Speech-based applications commonly come with web-based or printed manuals. Alternatively, the dialogue can be designed so that users should be able to start using the application on their own. We studied an alternative approach, an integrated tutor. The tutor participates in the interaction when new users learn to use a speech-based system. It teaches the users how to operate the system and monitors user actions to be certain that the users do indeed learn. In this paper we describe our experiences with the design and the iterative development of an integrated tutor. Expert evaluation and two user tests were conducted with different versions of the tutor. The results show that the tutor can effectively guide new users. We identify the six most important lessons learned, the most important being that it is essential to spot problems by monitoring user actions, especially when novice users are tutored.

There is demand for subjective metrics in spoken dialogue system evaluation. SERVQUAL is a service quality evaluation method developed by marketing academics. It produces a subjective measure of the gap between expectations and perceptions in five service quality dimensions common for all services. We present how the method was applied to spoken dialogue system evaluation. In order to improve the suitability of the original method, we modified the test questionnaire and the test process. We demonstrate how the modified method was successfully used in an evaluation of a telephone-based e-mail application. The evaluation gave us directions for further development of the system. In addition, we found some interesting phenomena, such as the variation between genders. We present how the method can be further improved, for example, by dividing the questionnaire into two parts.

We present how robustness and adaptivity can be supported by the spoken dialogue system architecture. AthosMail is a multilingual spoken dialogue system for e-mail domain. It is being developed in the EU-funded DUMAS project. It has flexible system architecture supporting multiple components for input interpretation, dialogue management and output generation. In addition to language differences, these components have great variation in their approaches for spoken interaction. For example, the dialogue management components implement different dialogue control models. The system architecture makes the interaction robust and adaptive by utilizing different approaches for spoken interaction in a single application by selecting suitable components at runtime. 1.

In this paper, we introduce the concept of integrated tutoring in speech applications. An integrated tutoring system teaches the use of a system to a user while he/she is using the system in a typical manner. Furthermore, we introduce the general principles of how to implement applications with integrated tutoring agents and present an example implementation for an existing e-mail system. The main innovation of the approach is that the tutoring agents are part of the application, but implemented in a way which makes it possible to plug them into the system without modifying it. This is possible due to a set of small, stateless agents and a shared Information Storage provided by our system architecture. Integrated tutoring agents are easily expandable and configurable, and general agents can be shared between applications. We have also received positive feedback about integrated tutoring in initial user tests conducted with the implementation.

In this paper, we introduce an architecture for a new generation of speech applications. The presented architecture is based on our previous work with multilingual speech applications and extends it by introducing support for synchronized distributed dialogues, which is needed in emerging application areas, such as mobile and ubiquitous computing. The architecture supports coordinated distribution of dialogues, concurrent dialogues, system level adaptation and shared system context. The overall idea is to use interaction agents to distribute dialogues, use an evaluation mechanism to make them dynamically adaptive and synchronize them by using a coordination mechanism with triggers and transactions. We present experiences from several applications written on top of the freely available architecture.