Navigating complex environments is crucial for human life, yet understanding the cognitive processes involved in its wayfinding component remains challenging. One theoretical model that explains these processes is Downs and Stea’s four-step model. Our study builds on this model to empirically analyze its steps, focusing particularly on the monitoring step. Machine learning models were trained on gaze behavior and head/body movement data from over 300 routes walked by 56 participants in a real-world outdoor study, predicting three of these wayfinding steps: self-localization, route planning, and goal recognition. Applying this trained model to the respective monitoring segment of the same routes suggests that monitoring includes micro-versions of these three steps, indicating it operates as a recursive process rather than a distinct cognitive step. By bridging theoretical frameworks with empirical evidence, these findings enhance our understanding of spatial cognition and can inform the design of navigational tools and urban spaces.

RU-AI dataset is constructed based on three large publicly available datasets: Flickr8K, COCO, and Places205, by adding their corresponding machine-generated pairs.

The applicability of augmented reality (AR) as a tool for geospatial data analysis and urban environment interaction relies on developing robust and accurate systems capable of aligning the virtual reference frame with the geographic one. In this article, we introduce our work toward the conceptualization and realization of Geographic-Aware Augmented Reality (GeoAR), including an evaluated framework for the automatic registration of georeferenced AR content. The proposed framework uses a novel calibration method that enables highly accurate placement of augmentations at their assigned geographic coordinate. Moreover, it introduces four calibration approaches suitable for different user needs. The framework was evaluated to assess the robustness of the augmentation’s positional accuracy in three areas with different environmental characteristics, using references up to 50 m away while the user moves around. The results demonstrate that this framework supports novel outdoor AR applications, extending the possibilities in research and urban applications.

Navigation assistance systems have become integral to our daily routines, helping us to find our way through unfamiliar environments. However, their use may come at a price, as empirical evidence suggests a potentially harmful impact of these systems on our spatial abilities, including the acquisition of spatial knowledge. This could be remedied by giving users more freedom and involving them in the decision-making process. Therefore, we present a navigation system that combines augmented reality and Beeline Navigation (BeeAR). Here, the location of the destination is overlaid with a digital landmark and permanently displayed to the user via a visual, translucent AR display (without a map). Since the digital content is integrated into the real world, no mapping between the device and reality is required, potentially lowering the workload. Making one's own decisions along the route is expected to increase engagement with the environment, leading to increased acquisition of spatial knowledge. We compare BeeAR with findings from a previous study comparing Free Choice Navigation (FCN) and Turn-by-Turn (TBT) navigation conducted along the same routes on the outskirts of Vienna, Austria. Although BeeAR and FCN do not provide users with a map, BeeAR users could better retrace the walked route and remembered more points of interest along the route than FCN users. Participants of all three navigation conditions achieved a high configuration similarity between drawn points of interest and their true locations, albeit only one navigation condition included a map.

Saccadic eye movements are known to serve as a suitable proxy for tasks prediction. In mobile eye-tracking, saccadic events are strongly influenced by head movements. Common attempts to compensate for head-movement effects either neglect saccadic events altogether or fuse gaze and head-movement signals measured by IMUs in order to simulate the gaze signal at head-level. Using image processing techniques, we propose a solution for computing saccades based on frames of the scene-camera video. In this method, fixations are first detected based on gaze positions specified in the coordinate system of each frame, and then respective frames are merged. Lastly, pairs of consecutive fixations –forming a saccade- are projected into the coordinate system of the stitched image using the homography matrices computed by the stitching algorithm. The results show a significant difference in length between projected and original saccades, and approximately 37% of error introduced by employing saccades without head-movement consideration.

A novel eye-tracked measure of the frequency of pupil diameter oscillation is proposed for capturing what is thought to be an indicator of cognitive load. The proposed metric, termed the Index of Pupillary Activity, is shown to discriminate task difficulty vis-a-vis cognitive load (if the implied causality can be assumed) in an experiment where participants performed easy and difficult mental arithmetic tasks while fixating a central target (a requirement for replication of prior work). The paper's contribution is twofold: full documentation is provided for the calculation of the proposed measurement which can be considered as an alternative to the existing proprietary Index of Cognitive Activity (ICA). Thus, it is possible for researchers to replicate the experiment and build their own software which implements this measurement. Second, several aspects of the ICA are approached in a more data-sensitive way with the goal of improving the measurement's performance.

Complex information visualizations, such as thematic maps, encode information using a particular symbology that often requires the use of a legend to explain its meaning. Traditional legends are placed at the edge of a visualization, which can be difficult to maintain visually while switching attention between content and legend.

During pedestrian navigation in outdoor urban environments we often utilize assistance systems to support decision-making. These systems help wayfinders by providing relevant information withing the context of their surroundings, e.g., landmark-based instructions of the type "turn left at the church". Next to the instruction type and content, also the timing of the instruction must be considered in order to facilitate the wayfinding process. In this work we present our findings concerning the user and environmental factors that have an impact on the timing of instructions. We applied a survival analysis on data collected through an experiment in a realistic virtual environment in order to analyze the expected distance to the decision point until instructions are needed. The presented results can be used by navigation systems for instruction timing based on the characteristics of the current wayfinder and environment.

Location based services are a common application scenario in mobile and ubiquitous computing use. A typical issue with cartographic applications in this domain is the limited size of the displayed map, which makes interaction and visualization a difficult problem to solve. With the increasing popularity of head mounted displays for VR and AR systems, an opportunity is presented for map-based applications to overcome the limitation of the small display size, as the user's information visualization space can extend to his entire surroundings. In this paper we present a preliminary investigation into how interaction with such very large display maps can take place, using a virtual reality headset as the sole input and interaction method.

Location-based services (LBS) provide more useful, intelligent assistance to users by adapting to their geographic context. For some services that context goes beyond a location and includes further spatial parameters, such as the user’s orientation or field of view. Here, we introduce Gaze-Informed LBS (GAIN-LBS), a novel type of LBS that takes into account the user’s viewing direction. Such a system could, for instance, provide audio information about the specific building a tourist is looking at from a vantage point. To determine the viewing direction relative to the environment, we record the gaze direction relative to the user’s head with a mobile eye tracker. Image data from the tracker’s forward-looking camera serve as input to determine the orientation of the head w.r.t. the surrounding scene, using computer vision methods that allow one to estimate the relative transformation between the camera and a known view of the scene in real-time and without the need for artificial markers or additional sensors. We focus on how to map the point of regard of a user to a reference system, for which the objects of interest are known in advance. In an experimental validation on three real city panoramas, we confirm that the approach can cope with head movements of varying speed, including fast rotations up to to 63 degrees per second. We further demonstrate the feasibility of GAIN-LBS for tourist assistance with a proof-of-concept experiment in which a tourist explores a city panorama, where the approach achieved a recall that reaches over 99%. Finally, a GAIN-LBS can provide objective and qualitative ways of examining the gaze of a user based on what the user is currently looking at.

A convolution-filtering technique is introduced for the synthesis of eye gaze data. Its purpose is to produce, in a controlled manner, a synthetic stream of raw gaze position coordinates, suitable for: (1) testing event detection filters, and (2) rendering synthetic eye movement animations for testing eye tracking gaze estimation algorithms. Synthetic gaze data is parameterized by sampling rate, microsaccadic jitter, and simulated measurement error. Sampled synthetic gaze data is compared against real data captured by an eye tracker showing similar signal characteristics.

The focus of this study is on wayfinding in large complex buildings with different wayfinding devices. The interaction of pedestrians of such devices is always also interplay with the surrounding environment and its specific features. Furthermore different wayfinding assistances can elicit different needs for additional information from the environment to make accurate choices at decision points. We aim to shed light on how characteristics of decision points in combination with different wayfinding devices shape wayfinders' visual attention. 60 participants individually looked for three destinations in the same order. They navigated with 1) a printed map, 2) a digital map, or 3) without a map, only using full-coverage numeric signage. To gain first insights fixation frequencies on maps and signage as well as the correct and incorrect route options were recorded with a mobile eyetracker and analyzed for 28 decision points and four decision point categories. The results indicated that starting points play a special role in planning the route ahead. Furthermore points that allow for a floor change lead to a higher attention and information search.

Pedestrian navigation systems help us make a series of decisions that lead us to a destination. Most current pedestrian navigation systems communicate using map-based turn-by-turn instructions. This interaction mode suffers from ambiguity, its user's ability to match the instruction with the environment, and it requires a redirection of visual attention from the environment to the screen. In this paper we present GazeNav, a novel gaze-based approach for pedestrian navigation. GazeNav communicates the route to take based on the user's gaze at a decision point. We evaluate GazeNav against the map-based turn-by-turn instructions. Based on an experiment conducted in a virtual environment with 32 participants we found a significantly improved user experience of GazeNav, compared to map-based instructions, and showed the effectiveness of GazeNav as well as evidence for better local spatial learning. We provide a complete comparison of navigation efficiency and effectiveness between the two approaches.

Predicting the moment when a visual explorer of a place loses interest and starts to get bored is of considerable importance to the design of touristic information services. This paper investigates factors affecting the duration of the visual exploration of a city panorama. We report on an empirical outdoor eye tracking study in the real world with tourists following a free exploration paradigm without a time limit. As main result, the number of areas of interest revisited during a short period was found to be a good predictor for the total exploration duration.

The spatio-temporal characteristics of eye movements vary according to the activity the user of a cartographic map is performing. In this paper, we use these eye movement characteristics to automatically detect the map user's activity, an approach with great potential in gaze-assistive map interfaces. A dataset of 587 eye movement recordings from 17 participants was used to train and cross-validate a support vector machine (SVM) classifier over 229 features. The classifier can distinguish 6 common map activities with an accuracy of approx. 78%.

This paper introduces gaze map matching as the problem of algorithmically interpreting eye tracking data with respect to geographic vector features, such as a road network shown on a map. This differs from previous eye tracking studies which have not taken into account the underlying vector data of the cartographic map. The paper explores the challenges of gaze map matching and relates it to the (vehicle) map matching problem. We propose a gaze map matching algorithm based on a Hidden Markov Model, and compare its performance with two purely geometric algorithms. Two eye tracking data sets recorded during the visual inspection of 14 road network maps of varying realism and complexity are used for this evaluation.

Orientation on small display maps is often difficult because the visible spatial context is restricted. This paper proposes to provide the history of a user's visual attention on a map as visual clue to facilitate orientation. Visual attention on the map is recorded with eye tracking, clustered geo-spatially, and visualized when the user zooms out. This implicit gaze-interaction concept, called GeoGazemarks, has been evaluated in an experiment with 40 participants. The study demonstrates a significant increase in efficiency and an increase in effectiveness for a map search task, compared to standard panning and zooming.