This study investigates the effects of auditory and visual quality in virtual reality (VR) environments on cognitive load and engagement through pupillometric measurement. It provides a comprehensive consideration of their influence on physiological processes collectively. The experiment controlled both auditory (low-fidelity muffled, normal dynamic, and high-volume monotonous) and visual (less detailed, normal, and richly detailed) stimuli systematically in three VR museum environments. Pupil diameter measurements were employed as primary indicators of cognitive processing, with rigorous preprocessing to ensure data integrity. For visual conditions, low-quality settings with less detail consistently generated maximum stress responses, while normal-quality visuals were determined to optimize the balance of engagement and comfort. High-quality visuals in dim settings had slightly more engagement but had to be calibrated precisely to avoid overstimulation. The audio results indicated that normal-quality dynamic sound produced the most stable pupil responses across all visual contexts and served as a cognitive anchor. Poor-quality audio consistently degraded experience quality, supporting the fact that visual richness cannot compensate for poor auditory design. Notably, combining normal-quality audio with normal-quality visuals provided the optimal physiological profile. These findings guide theoretical advancement in multisensory integration in VR and offer design recommendations of utility: (1) prefer balanced modulation of the senses, (2) avoid visual extremes of simplicity with high luminance, and (3) seek harmonious mixtures of normal-fidelity sensory and visual content. Future research needs to explore individual differences and long-term adaptation in order to further refine these guidelines for a variety of educational, training, and therapy uses in VR.

While learning instruments playing, one engages in the following mental and motor activities; reading notes in a musical score, retrieving relevant information from long-term memory concerning how to play the notes, moving relevant body parts, and verifying his/her performance against the expected sound. The learner is required to coordinate these activities in the learning process. This paper assumes that how the process of coordination is carried out should depend on the degree of matching between the learner’s skill and the musical score which should appear in the process of reading; visually encoding the to-be-played notes and verifying the sound from the instrument he/she has just played. This paper proposes a method for objectively measuring the degree of matching by using eye movements data while reading a musical score with hearing notes.

Generic skills (GS) are meta-competencies necessary for acquiring knowledge from provided information in classes. This paper assumes that GS consists of a skill associated with the literacy for pulling information from the accessible contents in the classes, and a skill associated with the literacy for pushing knowledge stored in the students’ brain for obtaining useful information from the instructors. Based on the assumption, we propose a method for assessing developmental stages of GS. Observational studies were conducted in a real classroom for testing this assumption. For preparation, we defined a micro-structure of GS. Using the structure, we identified three critical parameters that should characterise the students’ performance on the micro-structure. Then, we selected elite monitors to verify whether micro-structure works in our study. The results suggested that the micro-structure of GS works well in classroom situations. We conclude that three critical parameters are good parameters for assessing developmental stages of GS.

Navigation behaviour is characterised by a strong interaction between the navigating agent (i.e. human beings) and the environment in which the navigation occurs. As the interaction is time critical, a theory for understanding the navigation behaviour must be capable of simulating navigation along the time dimension. This article introduces the Model Human Processor with Real-Time Constraints (MHP/RT; Toyota, M. and Kitajima, M., 2010a. MHP/RT: model human processor with real time constraints. In: S. Ohlsson and R. Catrambone, eds., Proceedings of the 32nd annual conference of the Cognitive Science Society. Austin, TX: Cognitive Science Society, 529.) as an architecture model for simulating the human navigation behaviour. MHP/RT was created by combining two seminal works in different fields that deal with human behaviour. The first work is the Model Human Processor (MHP) in the field of applied psychology (Card, S.K., Moran, T.P., and Newell, A., 1983. The psychology of human–computer interaction. Hillsdale, NJ: Lawrence Erlbaum Associates.). MHP successfully simulates the human users' operating information devices to accomplish computerised tasks. The other work focuses on Two Minds that operate in human beings' economical decisions (Evans, J.S.B.T. and Frankish, K., eds., 2009. In Two Minds: dual processes and beyond. Oxford: Oxford University Press.). This is the basis of the field of behaviour economics founded by D. Kahneman. This study demonstrates how MHP/RT simulates people's navigation behaviour by drawing a concrete example of navigation in a train station. The results of an observational study that was conducted independently of MHP/RT are introduced and further examined by running the processes defined in MHP/RT. The results of the observed navigation behaviour can be plausibly simulated by MHP/RT, confirming the validity of MHP/RT.

This paper reports an eye-tracking experiment conducted to compare alternative representations of directories typically shown on web pages in search of a best representation for deaf persons. The experiment simulated a directory-based information search task to understand how it is performed when directories are represented in text, labeled-pictograms, or unlabeled-pictograms.Twenty-one deaf and 21 hearing participants were asked to select one of 27 directories represented in one of the three alternative formats for each of 38 queries. The result demonstrated that only in the labeled-pictogram representation, the hearing group and the deaf group performed equally well in terms of the eye movement measures.

Finding information by successively selecting hyperlinks on web pages is a typical task performed on websites. A number of web usability studies have provided important insights about how web visitors carry out a search, and have concluded that "following information scent" is the fundamental process involved in the behavior. The purpose of this paper is to explore the relationship between the strength of information scent and web visitors'eye movements. Four web page types with different usability problems were considered. In an eyetracking experiment, eleven participants were asked to find an article on a simulated encyclopedia website by first selecting a heading from among nine provided headings, then selecting the appropriate topic link under the selected heading. The number of eye fixations, the duration of the fixations, and the task completion times were analyzed. The eye-tracking study reported in this paper added further insight to the knowledge gained from traditional web usability studies, in which visitors'performance are measured by the total number of clicks and task completion times. Website visitors'performance will not exhibit any differences in the initial heading selection stage irrespective of whether or not the pages have usability problems. However, performance will deteriorate in terms of the total number of fixations in the subsequent link selection stage when the web page has any kind of usability problem.

The World Wide Web has become an important source of information, as much as traditional media like books, newspapers, and television. While there have been many studies on Web searching, research into Web-viewing behavior using eye-tracking systems has only recently begun [Pan et al., 2004]. Josephson and Holmes [2002] studied Web-viewing behavior focusing on the category of Web page visual design. They suggested that eye movements were affected by the following two factors: (1) visual design of Web pages and (2) habitually preferred path across the visual stimuli. However, these previous studies did not sufficiently consider the user's experience. The purpose of this study is to investigate how past Web-browsing experience influences Web-viewing behavior. We used a detailed questionnaire to measure a user's Web-browsing experience and analyzed the eye-tracking data based on the user's prior Web experience.

Recently, with continued advances in information technology, there is an ever-growing amount of information accumulated on the World Wide Web. At the same time, the need to make this information accessible to any person who needs it becomes a serious issue. This paper focuses on web content accessibility for the hard-of-hearing. This is motivated by the fact that the first author, who is engaged in educating hard-of-hearing persons through daily classes, felt that hard-of-hearing persons would interact with web pages differently than hearing persons. Using web-based interactive course materials seems effective since they allow the creator to control the presentation of the content. However, the issue of how the hard-of-hearing interact with the web has not been adequately studied.

The Cognitive Walkthrough for the Web (CWW) is a partially automated usability evaluation method for identifying and repairing website navigation problems. Building on five earlier experiments [3,4], we first conducted two new experiments to create a sufficiently large dataset for multiple regression analysis. Then we devised automatable problem-identification rules and used multiple regression analysis on that large dataset to develop a new CWW formula for accurately predicting problem severity. We then conducted a third experiment to test the prediction formula and refined CWW against an independent dataset, resulting in full cross-validation of the formula. We conclude that CWW has high psychological validity, because CWW gives us (a) accurate measures of problem severity, (b) high success rates for repairs of identified problems (c) high hit rates and low false alarms for identifying problems, and (d) high rates of correct rejections and low rates of misses for identifying non-problems.

In human-human communications, especially in face-to-face communication, sub-verbal and non-verbal messages have more importance than messages transported by words in communications. On the other hand, in traditional man-machine interfaces, machines only understand pre-defined operations, and never understand operators' sub-verbal and non-verbal messages. This causes some difficulty of machine usability. Grasping-and-moving are fundamental hand operations necessary for performing tasks using modern man-machine interfaces. However, behavioral information associated with grasping, such as force, posture, etc., has not been utilized for traditional interfaces. Although behavioral information should have potential utility in developing task-adaptive and operator-adaptive interfaces; it is known that how people grasp devices is dependent on the purpose of these tasks. As an example of typical pointing devices, This work describes a computer mouse whose approaching speed, or gain, is adjusted depending on the nature of the task. This work suggests that the grasping-and-moving tasks should consist of two phases. The first or approaching phase is when the operator moves the mouse pointer close to the target with a small grasping force. The second or positioning phase follows when the operator makes a fine adjustment of the pointer to locate it within the target area using a larger grasping force. This proposed new computer mouse has a sensor to detect the operator's grasping force and uses it to control the gain of the mouse. Two sets of experiments were conducted; one for confirming the assumption that the grasping force should be small while approaching the target and large while positioning, and the other for confirming that the proposed mouse should be effective in reducing the time necessary for pointing various kinds of objects.

Methods for identifying usability problems in web page designs should ideally also provide practical methods for repairing the problems found. Blackmon et al. [2] proved the usefulness of the Cognitive Walkthrough for the Web (CWW) for identifying three types of problems that interfere with users' navigation and information search tasks. Extending that work, this paper reports a series of two experiments that develop and prove the effectiveness of both full-scale and quick-fix CWW repair methods. CWW repairs, like CWW problem identification, use Latent Semantic Analysis (LSA) to objectively estimate the degree of semantic similarity (information scent) between representative user goal statements (100-200 words) and heading/link texts on each web page. In addition to proving the effectiveness of CWW repairs, the experiments reported here replicate CWW predictions that users will face serious difficulties if web developers fail to repair the usability problems that CWW identifies in web page designs [2].

We propose a new approach for indexing Web contents. The essence of our approach is that we use a psycho-linguistic word database for calculating the index of Web pages. Since there is no existing database designed for this purpose, we started our study by creating a word database. We will show that the database can be effectively used for estimating the reading levels of specific Web pages. We will also show that this approach can be used to reflect user profiles in Web searches.

This paper proposes a transformation of the Cognitive Walkthrough (CW), a theory-based usability inspection method that has proven useful in designing applications that support use by exploration. The new Cognitive Walkthrough for the Web (CWW) is superior for evaluating how well websites support users' navigation and information search tasks. The CWW uses Latent Semantic Analysis to objectively estimate the degree of semantic similarity (information scent) between representative user goal statements (100-200 words) and heading/link texts on each web page. Using an actual website, the paper shows how the CWW identifies three types of problems in web page designs. Three experiments test CWW predictions of users' success rates in accomplishing goals, verifying the value of CWW for identifying these usability problems

The linked model of comprehension-based action planning and instruction taking (LICAI) simulates performing by exploration tasks using applications hosted on systems with graphical user interfaces. The tasks are given to the user as written exercises containing no information about the correct action sequences. LICAI's comprehension and action-planning processes are based on Kintsch's construction-integration (C-I) theory for text comprehension. The model assumes that comprehending instructions is a strategic process; instruction texts must be elaborated using specialized strategres that guide goal generation. LICAI comprehends the instructions and generates goals that are then stored in memory. The action-planning processes are controlled by goals retrieved from memory. Representations of goals that can guide exploration are restricted by the C-I architecture. The model predicts that successful exploration requires linking of the goal representation with the label on the correct object. The model is evaluated by comparing its predictions with results from an experimental study of learning by exploration by Franzke (1994, 1995). We discuss the implications of LICAI for designing instruction materials and interfaces that facilitate exploration.

Article A comprehension-based model of exploration Share on Authors: Muneo Kitajima National Institute of Bioscience and Human-Technology, 1-1 Higashi Tsukuba Ibaraki 305, Japan National Institute of Bioscience and Human-Technology, 1-1 Higashi Tsukuba Ibaraki 305, JapanView Profile , Peter G. Polson Institute of Cognitive Science, University of Colorado, Boulder, CO Institute of Cognitive Science, University of Colorado, Boulder, COView Profile Authors Info & Claims CHI '96: Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsApril 1996 Pages 324–331https://doi.org/10.1145/238386.238536Online:13 April 1996Publication History 11citation611DownloadsMetricsTotal Citations11Total Downloads611Last 12 Months5Last 6 weeks0 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

This paper describes a computational model of skilled use of a graphical user interface based on Kintsch's construction-integration theory [4, 8]. The model uses knowledge of a detailed representation of information on the display, a user's goals and expectations, knowledge about the interface, and knowledge about the application domain to compute actions necessary to accomplish the user's current goal. The model provides a well-motivated account of one kind of errors, action slips [14], made by skilled users. We show how information about the intermediate state of a task on the display plays a critical role in skilled performance, i.e., display-based problem solving.