Abstract In Virtual Reality (VR), locomotion methods allow users to adjust their location in the virtual environment. These methods are sometimes used in 3D-VR medical image manipulation, which has gained interest in VR in the medical field due to the immersive environment and increased number of possible interaction techniques. However, the medical 3D-VR image manipulation context differs from typical VR locomotion, as the focus is not on the navigation of the VR space, but rather on the point of view of the user to the 3D image. For this study, we recruited 24 participants to find and observe easy targets from simplified medical images. We compared whether the three VR locomotion methods, Diving, Grabbing and Teleporting, were suitable for medical 3D image manipulation tasks. Diving was found to be significantly more successful than Teleporting while being an equally fast method. Locomotion method preferences varied. Therefore, a VR system is suggested to provide various manipulation methods for the user. However, the methods need development before they can be used with radiologists for actual medical image analysis tasks. We fill a research gap from the VR medical image manipulation context, where even though VR locomotion methods, such as Teleporting have been sometimes used, there has been a lack of interaction technique studies of these methods. Therefore, we provide necessary data on locomotion methods for the designers and developers of 3D-VR medical image applications. This is an initial study to validate different techniques for medical image analysis tasks in VR.

To effectively utilize a gaze tracker in user interaction it is important to know the quality of the gaze data that it is measuring. We have developed a method to evaluate the accuracy and precision of gaze trackers in virtual reality headsets. The method consists of two software components. The first component is a simulation software that calibrates the gaze tracker and then performs data collection by providing a gaze target that moves around the headset's field-of-view. The second component makes an off-line analysis of the logged gaze data and provides a number of measurement results of the accuracy and precision. The analysis results consist of the accuracy and precision of the gaze tracker in different directions inside the virtual 3D space. Our method combines the measurements into overall accuracy and precision. Visualizations of the measurements are created to see possible trends over the display area. Results from selected areas in the display are analyzed to find out differences between the areas (for example, the middle/outer edge of the display or the upper/lower part of display).

Vibrotactile feedback is widely used in mobile devices because it provides a discreet and private feedback channel. Gaze-based interaction, on the other hand, is useful in various applications due to its unique capability to convey the focus of interest. Gaze input is naturally available as people typically look at things they operate, but feedback from eye movements is primarily visual. Gaze interaction and the use of vibrotactile feedback have been two parallel fields of human–computer interaction research with a limited connection. Our aim was to build this connection by studying the temporal and spatial mechanisms of supporting gaze input with vibrotactile feedback. The results of a series of experiments showed that the temporal distance between a gaze event and vibrotactile feedback should be less than 250 ms to ensure that the input and output are perceived as connected. The effectiveness of vibrotactile feedback was largely independent of the spatial body location of vibrotactile actuators. In comparison to other modalities, vibrotactile feedback performed equally to auditory and visual feedback. Vibrotactile feedback can be especially beneficial when other modalities are unavailable or difficult to perceive. Based on the findings, we present design guidelines for supporting gaze interaction with vibrotactile feedback.

In gesture-based user interfaces, the effort needed for learning the gestures is a persistent problem that hinders their adoption in products. However, people's natural gaze paths form shapes during viewing. For example, reading creates a recognizable pattern. These gaze patterns can be utilized in human-technology interaction. We experimented with the idea of inducing specific gaze patterns by static drawings. The drawings included visual hints to guide the gaze. By looking at the parts of the drawing, the user's gaze composed a gaze gesture that activated a command. We organized a proof-of-concept trial to see how intuitive the idea is. Most participants understood the idea without specific instructions already on the first round of trials. We argue that with careful design the form of objects and especially their decorative details can serve as a gaze-based user interface in smart homes and other environments of ubiquitous computing.

Exploratory analysis of gaze data requires methods that make it possible to process large amounts of data while minimizing human labor. The conventional approach in exploring gaze data is to construct heatmap visualizations. While simple and intuitive, conventional heatmaps do not clearly indicate differences between groups of viewers or give estimates for the repeatability (i.e., which parts of the heatmap would look similar if the data were collected again). We discuss difference maps and significance maps that answer to these needs. In addition we describe methods based on automatic clustering that allow us to achieve similar results with cluster observation maps and transition maps. As demonstrated with our example data, these methods are effective in highlighting the strongest differences between groups more effectively than conventional heatmaps.

Selecting an object is a basic interaction task in virtual reality (VR) environments. Interaction techniques with gaze pointing have potential for this elementary task. There appears to be little empirical evidence concerning the benefits and drawbacks of these methods in VR. We ran an experiment studying three interaction techniques: ray casting, dwell time and gaze trigger, where gaze trigger was a combination of gaze pointing and controller selection. We studied user experience and interaction speed in a simple object selection task. The results indicated that ray casting outperforms both gaze-based methods while gaze trigger performs better than dwell time.

Interactions between humans and virtual environments rely on timely and consistent sensory feedback, including haptic feedback. However, many questions remain open concerning the spatial location of haptics on the user's body in VR. We studied how simple vibrotactile collision feedback on two less studied locations, the temples, and the wrist, affects an object picking task in a VR environment. We compared visual feedback to three visual-haptic conditions, providing haptic feedback on the participants' (N=16) wrists, temples or simultaneously on both locations. The results indicate that for continuous, hand-based object selection, the wrist is a more promising feedback location than the temples. Further, even a suboptimal feedback location may be better than no haptic collision feedback at all.

Immersive 360° video needs new ways of interaction. We compared three different interaction methods to find out which one of them is the most applicable for controlling 360° video playback. The compared methods were: remote control, pointing with head orientation, and hand gestures. A WebVR-based 360° video player was built for the experiment.

In a study with 12 participants we compared two smooth pursuit based widgets and one dwell time based widget in adjusting a continuous value. The circular smooth pursuit widget was found to be about equally efficient as the dwell based widget in our color matching task. The scroll bar shaped smooth pursuit widget exhibited lower performance and lower user ratings.

The best way to construct user interfaces for smart glasses is not yet known. We investigated the use of eye tracking in this context in two experiments. The eye and head movements were combined so that one can select the object to interact by looking at it and then change a setting in that object by turning the head horizontally. We compared three different techniques for mapping the head turn to scrolling a list of numbers with and without haptic feedback. We found that the haptic feedback had no noticeable effect in objective metrics, but it sometimes improved user experience. Direct mapping of head orientation to list position is fast and easy to understand, but the signal-to-noise ratio of eye and head position measurement limits the possible range. The technique with constant rate of change after crossing the head angle threshold was simple and functional, but slow when the rate of change is adjusted to suit beginners. Finally the rate of change dependent on the head angle tends to lead to fairly long task completion times, although in theory it offers a good combination of speed and accuracy.

Virtual and physical worlds are merging. Currently users of head-mounted displays cannot have unobtrusive tactile feedback while touching virtual objects. We present a mid-air tactile feedback system for head-mounted displays. Our prototype uses the focus of a modulated ultrasonic phased array for unobtrusive mid-air tactile feedback generation. The array and the hand position sensor are mounted on the front surface of a head-mounted virtual reality display. The presented system can enhance 3D user interfaces and virtual reality in a new way.

Anticipating the emergence of gaze tracking capable mobile devices, we are investigating the use of gaze as an input modality in handheld mobile devices. We conducted a study of combining gaze gestures with vibrotactile feedback. Gaze gestures were used as an input method in a mobile device and vibrotactile feedback as a new alternative way to give confirmation of interaction events. Our results show that vibrotactile feedback significantly improved the use of gaze gestures. The tasks were completed faster and rated easier and more comfortable when vibrotactile feedback was provided.

Consistent measuring and reporting of gaze data quality is important in research that involves eye trackers. We have developed TraQuMe: a generic system to evaluate the gaze data quality. The quality measurement is fast and the interpretation of the results is aided by graphical output. Numeric data is saved for reporting of aggregate metrics for the whole experiment. We tested TraQuMe in the context of a novel hidden calibration procedure that we developed to aid in experiments where participants should not know that their gaze is being tracked. The quality of tracking data after the hidden calibration procedure was very close to that obtained with the Tobii's T60 trackers built-in 2 point, 5 point and 9 point calibrations.

Eye tracking input often relies on visual and auditory feedback. Haptic feedback offers a previously unused alternative to these established methods. We describe a study to determine the natural time limits for haptic feedback to gazing events. The target is to determine how much time we can use to evaluate the user gazed object and decide if we are going to give the user a haptic notification on that object or not. The results indicate that it is best to get feedback faster than in 250 milliseconds from the start of fixation of an object. Longer delay leads to increase in incorrect associations between objects and the feedback. Delays longer than 500 milliseconds were confusing for the user.

People use social-photography services to tell stories about themselves and to solicit responses from viewers. State of the-art services concentrate on textual comments, "Like" buttons, or similar means for viewers to give explicit feedback, but they overlook other, non-textual means. This paper investigates how emotion responses--as video clips captured by the front camera of a cell phone and used as tags for the individual photo viewed--can enhance photo-sharing experiences for close-knit groups. Our exploration was carried out with a mobile social-photography service called Social Camera. Four user groups (N=19) used the application for two to four weeks. The study's results support the value of using front-camera video recordings to glean emotion response. It supports lightweight phatic social interactions not possible with comments and "Like" buttons. Most users kept sharing emotion responses throughout the study. They typically shared the responses right after they saw a just taken photo received from a remote partner. They used the responses to share their current contexts with others just as much as to convey nuanced feelings about a photo. We discuss the implications for future design and research.

Adaptation is an effective method for improving classification accuracy and a committee structure can in general improve on its members' performance. Therefore an adaptive committee structure is a tempting approach. Rejection may be used in handwriting recognition to improve performance through either directing the problematic character to a special classifier that handles such hard cases or discarding it. The experiments in this paper compare several fundamentally different approaches to implementing rejection in an adaptive committee classifier. A dynamically expanding context (DEC) - based committee is used for evaluating these approaches. The results show that if the rejected classes are handled with a 50% error rate, the performance is improved. A scheme in which there is an adjustable threshold for distance-based rejection is an effective method for implementing rejection in this setting.

This work describes a prototype-based online handwritten character recognition system and a two-phase recognition scheme aimed to speed up the recognition. In the first phase, the prototype set is pruned and ordered on the basis of preclassification performed with heavily down-sampled characters and prototypes. In the second phase, the final classification is performed without down-sampling by using the reduced set of prototypes. Two down-sampling methods, a linear and nonlinear one, have been analyzed to see their properties regarding the recognition time and accuracy.

This paper proposes a method of identifying character-like regions in order to extract and recognize characters in natural color scene images automatically. After connected component extraction based on a multi-group decomposition scheme, alignment analysis is used to check the block candidates, namely, the character-like regions in each binary image layer and the final composed image. Priority adaptive segmentation (PAS) is implemented to obtain accurate foreground pixels of the character in each block. Then some heuristic meanings such as statistical features, recognition confidence, and alignment properties, are employed to justify the segmented characters. The algorithms are robust for a wide range of character fonts, shooting conditions, and color backgrounds. Results of our experiments are promising for real applications.

Because of the lack of prior knowledge about the color of characters and the disturbance caused by lighting conditions and noise, character segmentation from scene images is a very difficult issue. In this paper, a novel character segmentation method for color images from a digital camera is presented, combining edge detection, a watershed transform and clustering. Since the characters extracted from color images are output in a binary format, they can be input directly to an OCR system for recognition. Experimental results have proven the method's effectiveness.

The paper deals with the problem of finding good prototypes for a condensed nearest neighbor classified in a recognition system. A comparison study is done between two prototype representation schemes. The prototype search is done by a genetic algorithm which is able to generate novel prototypes (i.e. prototypes which are not among the training samples). It is shown that the generalized representation scheme is more powerful, giving significantly larger normalized interclass distances. It is also shown that both representation schemes with generic algorithm give significantly better prototypes than a direct prototype selection algorithm, which can select only among the training samples.

Methods for controlling the adaptation process of an online handwritten character recognizer are studied. The classifier is based on the k-nearest neighbor rule and it is adapted to a new writing style by adding new prototypes, deactivating confusing prototypes, and reshaping existing prototypes in a self-supervised fashion. The dissimilarity measure used for the comparison of characters is a nonlinear curve matching method base on dynamic time warping algorithm. Time needed for the evaluation of the dissimilarity measure for a single character depends linearly on the size of the prototype set. The purpose of the control methods is to increase the classifier's tolerance to malformed or mislabelled learning samples and to limit the growth of the prototype set. The control methods either set an upper limit for the number of prototypes per class or switch the adaptation of a particular character class on or off depending on the earlier performance of the classifier.

We have developed an adaptive handwriting recognizer for isolated Latin characters in which the adaptive behavior is based on the dynamically expanding context (DEC) algorithm. In our current system, the outputs of a set of static classifiers are combined in a committee machine, whose rules are adapted. Every misclassified character gives rise to adding a new DEC rule to the rule set of the committee. When the existing rules fail to produce a correct recognition output, more and more context information is utilized in forming the new DEC rules. Not only the first-ranking outputs from the member classifiers but also the second-ranking ones can be taken into account when forming the DEC rules. In the experiments described in this paper, various options in the implementation of the DEC committee classifier are evaluated. The results of the experiments show that the system is capable of fast adaptation to the user's handwriting and lead to lowered recognition error rates.

We have developed an adaptive online recognizer that is suitable for recognizing isolated alphanumeric characters. It is based on the k nearest neighbor rule. Various dissimilarity measures, all based on dynamic time warping (DTW), have been studied. The main focus of this work is on online adaptation. The adaptation is performed by modifying the prototype set of the classifier according to its recognition performance and the user's writing style. These adaptations include: (1) adding new prototypes, (2) inactivating confusing prototypes, and (3) reshaping existing prototypes. The reshaping algorithm is based on learning vector quantization (LVQ). The writers are allowed to use their own natural style of writing, and the adaptation is carried out during normal use in a self-supervised fashion and thus remains otherwise unnoticed by the user.

Subsystems for online recognition of handwriting are needed in personal digital assistants (PDA) and other portable handheld devices. We have developed a recognition system which enhances its accuracy by applying continuous adaptation to the user's writing style. The forms of adaptation we have experimented with take place simultaneously with the normal operation of the system and therefore, there is no need for separate training period of the device. The present implementation uses dynamic time warping (DTW) in matching the input characters with the stored prototypes. The DTW algorithm implemented with dynamic programming (DP) is, however both time and memory consuming. In our current research we have experimented with methods that transform the elastic templates to pixel images which can then be recognized by using statistical or neural classification. The particular neural classifier we have used is the local subspace classifier (LSC) of which we have developed an adaptive version.

Vector quantization (VQ) can be used to compress images with high compression ratios. The VQ methods produce a sequence of code values which identifies the codebook model vectors to be used as blocks of pixels in the decoded image. In this paper we define a novel non-lossy and computationally efficient method to further compress the code sequence based on the relative frequencies of the code values, and the spatial distribution of each code. In an example case we reduced the bit rate by 29%. A further reduction of 7 percentage units was obtained when the VQ codebook was produced by the self-organizing map (SOM) algorithm. A SOM codebook has the property that similar blocks have similar codes, which was used to take advantage of spatial redundancies in the image.

The self-organizing map (SOM) method is a new, powerful software tool for the visualization of high-dimensional data. It converts complex, nonlinear statistical relationships between high-dimensional data into simple geometric relationships on a low-dimensional display. As it thereby compresses information while preserving the most important topological and metric relationships of the primary data elements on the display, it may also be thought to produce some kind of abstractions. The term self-organizing map signifies a class of mappings defined by error-theoretic considerations. In practice they result in certain unsupervised, competitive learning processes, computed by simple-looking SOM algorithms. Many industries have found the SOM-based software tools useful. The most important property of the SOM, orderliness of the input-output mapping, can be utilized for many tasks: reduction of the amount of training data, speeding up learning nonlinear interpolation and extrapolation, generalization, and effective compression of information for its transmission.

The authors demonstrate that the self-organizing maps (SOMs) of Kohonen can be used as speech feature extractors that are able to take temporal context into account. They have investigated two alternatives for using SOMs as such feature extractors, one based on tracing the location of highest activity on a SOM, the other on integrating the activity of the whole SOM for a period of time. The experiments indicated that an improvement is achievable by using these methods.>

Two modifications of the self-organizing map (SOM) are proposed that, unlike the original algorithm, take into account time-dependent features of the input signal. In the first, a time average of a sequence of responses of one SOM is found, and this is recognized by another SOM. In the second, successive input patterns are concatenated together and recognized by the SOM. Comparing the results to those of a recognition system utilizing the original SOM, it was found that one could improve the recognition of isolated phonemes from 10.4% of errors to 7.0% and 5.0% of errors for the integration model and concatenation model, respectively. The improvement in a full-scale system where phoneme segments are also to be located is from 9.2% of errors to 8.2% and 7.6% of errors for the new methods, respectively.>

Three related possibilities for representing the sequential aspect of data using the self-organizing map model are studied. The quantitative results of experiments with artificial test data are described, and the most promising solutions for future work are discussed. In the first model, the backwards exponentially averaged input vectors are used as the pattern vector. In the second model, a concatenation model where actual input patterns from previous time slots are concatenated together to form a long pattern vector is used. Then the history is explicitly shown in the input vector. In the third model, an averaging scheme is again used, but with one map to get a first-order representation of the input data. The averaged responses from the first map are used as input patterns for the second map. Thus, the third model consists of a hierarchical structure of maps. It is concluded that the third system is the most interesting because of its accuracy, high tolerance to increasing noise, and high tolerance to the variation of the weighting parameters of the systems

Self-organizing maps have a bearing on traditional vector quantization. A characteristic that makes them more closely resemble certain biological brain maps, however, is the spatial order of their responses, which is formed in the learning process. A discussion is presented of the basic algorithms and two innovations: dynamic weighting of the input signals at each input of each cell, which improves the ordering when very different input signals are used, and definition of neighborhoods in the learning algorithm by the minimal spanning tree, which provides a far better and faster approximation of prominently structured density functions. It is cautioned that if the maps are used for pattern recognition and decision process, it is necessary to fine tune the reference vectors so that they directly define the decision borders.

A microprocessor-based real-time speech recognition system is described. It is able to produce orthographic transcriptions for arbitrary words or phrases uttered in Finnish or Japanese. It can also be used as a large-vocabulary isolated word recognizer. The acoustic processor of the system transcribing speech into phonemes is based on neural network principles. The so-called phonotopic maps constructed by a self-organizing process are employed. The coarticulation effects in phonetic transcriptions are compensated by means of automatically derived rules which describe the morphology of errors at the acoustic processor output. Without applying any language model, the recognition result is correct up to 92 or even 97 per cent referring to individual letters.>