Using pre-trained large language models (LLMs) as a backbone for time series prediction has recently attracted growing research interest. Existing approaches typically split time series into patches, map them to the token space of LLMs via a Tokenizer, process the tokens through a frozen or fine-tuned LLM backbone, and then reconstruct numerical forecasts using a Detokenizer. However, the actual effectiveness of LLMs for time series forecasting remains under debate. We observe that when trained and evaluated on small datasets, the Tokenizer–Detokenizer components often overfit to the specific data distribution, thereby masking the intrinsic predictive capability of the LLM backbone. To investigate the inherent potential of LLMs in this context, we design three models with identical architectures but distinct pre-training strategies. By leveraging large-scale pre-training, we obtain more unbiased Tokenizer–Detokenizer pairs that are seamlessly integrated with the LLM backbone. Through controlled experiments, we evaluate the zero-shot and few-shot forecasting performance of the LLM, offering insights into its true capabilities. Our extensive experiments reveal that, although the LLM backbone shows some promise, its performance remains limited and does not consistently surpass that of models specifically trained on large-scale time series data. Our source code is publicly available in the repository: https://github.com/SiriZhang45/LLM4TS.

As user experience (UX) increasingly shapes the value and usability of digital products, enabling non-designers to transform their ideas into coherent UX outcomes remains an important yet challenging problem. Despite advances in AI-assisted tools, UX design still requires cross-stage reasoning across strategic, structural, and interface layers, which is often unsupported by existing systems. To address this gap, this study proposes MAUX, a layered multi-agent UX framework that structures and coordinates design reasoning through goal-driven workflows. MAUX operationalizes the five classical UX layers (strategy, scope, structure, skeleton, and surface) via LLM-driven agents grounded in human–computer interaction principles. A blackboard architecture coordinated by a Meta-Agent maintains semantic consistency across stages, enabling systematic progression from initial intent to interface realization. The framework is demonstrated through the design of Urban AirLink, a conceptual low-altitude mobility platform, and evaluated against four LLM-based baselines in a controlled user study ( N = 50 ). Results show that MAUX significantly improves design quality, structural clarity, and cross-layer goal alignment compared to baseline approaches. These findings highlight the potential of MAUX to support reasoning-centered and democratized AI-assisted UX design.

Miscommunications between air traffic controllers (ATCOs) and pilots can lead to severe accidents, making accurate readback verification essential. Audio-visual interaction, namely generative subtitles, is expected to enhance readback verification by providing redundant information. However, this redundancy may also increase ATCOs’ cognitive workload and challenge their working memory capacity. This study investigated the effect of subtitles on readback performance and cognitive workload across different working memory capacities. A simulated verification task based on ATCO task analysis was employed, and functional near-infrared spectroscopy with eye-tracking was analyzed to assess subtitles’ effects. Results showed that subtitles improved readback performance across working memory groups without increasing cognitive workload. Oxygenated hemoglobin deactivation in frontal, parietal, and occipital regions indicated more efficient cognitive processing, and higher blink rates among high working memory participants suggested reduced cognitive load. This study deepens understanding of audio-visual interaction and offers practical insights for enhancing ATCO–pilot communication.

Cognitive support in aviation operations can help reduce cognitive workload and improve flight safety. Existing data-driven models, typically trained for specific aircraft types, often lack generalization capabilities and cannot be directly applied to other aircrafts. The multimodal large language models (MLLMs) have shown significant value in various applications without additional training. Motivated by the success of MLLMs, we explore and develop a proactive aviation cognitive support framework (PACS) based on MLLMs to proactively assist pilots in anomaly perception and decision-making during aviation emergencies. We propose a localization-augmented anomaly perception method to address the limitations of MLLMs in recognizing small-sized warning message text. To improve the accuracy and efficiency in decision-making, we design a hierarchically structured aviation knowledge base and a "Retrieval-Selection Generation" strategy to generate accurate operational instructions. The experimental results demonstrate high accuracy in anomaly perception (87.51%) and decision-making (93.04%), while reducing token count by up to 56.77%. Our PACS is expected to adapt to various scenarios without additional fine-tuning, offering a new paradigm for future human-AI collaboration in aviation operations.

In recent years, the potential applications of Large Multimodal Models (LMMs) in fields, such as healthcare, social psychology, and industrial design have attracted wide research attention, providing new directions for human factors research. For instance, LMM-based smart systems have become novel research subjects of human factors studies, and LMM introduces new research paradigms and methodologies to this field. Therefore, this article aims to explore the applications, challenges, and future prospects of LMM in the domain of human factors and ergonomics through an expert-LMM collaborated literature and patent review. Specifically, this article proposes a novel review method and introduces research studies and patents related to LMM-based accident analysis, human modeling, and intervention design. Subsequently, the article discusses future trends in research paradigm and challenges of human factors and ergonomics studies in the era of LMMs. It is expected that the review results offer valuable insights and serve as a reference for using LMMs in human factors research.

The integration of digital towers in air traffic control (ATC) intensifies visual complexity of controllers, increasing the risk of detection failure (DF) to warnings and compromising airspace safety. The inherent variability in human situational awareness and behaviors further complicates the differentiation and recognition of various DFs. This study deciphers DF by categorizing it into types based on Endsley’s situation awareness theory, identifying specific causes and key indicators. A four-phase framework—DF classification, DF induction experiment, gaze dynamics analytics, and DF-type recognition—was applied to gaze data from 26 subjects. Results revealed distinct gaze patterns for non-perception, unaware perception, and aware perception of warnings, with continuous warnings weakening operators’ awareness but enhancing foresight of warning implications. A random forest model achieved 80% precision in DF-type recognition, offering empirical support for real-time DF recognition and targeted interventions to improve visual warning detection and human-computer interaction in aviation safety.

Visual detection failures are a critical challenge in air traffic control (ATC), where undetected alerts can compromise operational safety and decision-making. Previous studies have primarily assessed detection failures through target fixation patterns, yet this method struggles to identify the more complex “look-but-fail-to-see” and “see-without-looking” scenarios. This underscores the necessity of exploring peripheral vision mechanisms, where dynamic tracking trajectories could better capture the scope of visual attention. Therefore, this study proposes a classification framework for visual detection by integrating peripheral vision tracking and human attentional states, including detection failures such as peripheral vision neglect and look-but-fail-to-see errors. A hierarchical detection failure recognition framework specific to the ATC settings is further developed and validated through an ATC simulation experiment. The framework first employs an Adaptive Symbolic Alert Detection method to identify and annotate ATC-specific alert regions with spatiotemporal uncertainty (achieving 95.24% precision), followed by LLM-based evaluation of operators’ visual attention to these regions to intelligently assign classification labels. Additionally, we introduce a fixation coordinate-sensitive multi-domain feature set that captures spatiotemporal and frequency-domain characteristics across detection types, achieving 93.13% four-class classification accuracy, outperforming traditional feature sets (83.69%) and both single-and dual-domain features (ranging from 76.82% to 90.11% accuracy). These findings demonstrate that our framework effectively captures a broader and structured range of visual detection failures, providing critical insights to improve the reliability of alert detection in ATC and the design of an intelligent human-centered ATC support system.

Long-term time series forecasting (LTSF) aims to predict future values for a long time based on historical data. The period term is an essential component of the time series, which is complex yet important for LTSF. Although existing studies have achieved promising results, they still have limitations in modeling dynamic complicated periods. Most studies only focus on static periods with fixed time steps, while very few studies attempt to capture dynamic periods in the time domain. In this paper, we dissect the original time series in time and frequency domains and empirically find that changes in periods are more easily captured and quantified in the frequency domain. Based on this observation, we propose to explore dynamic period features using rotation in the frequency domain. To this end, we develop the frequency-based rotation network (FRNet), a novel LTSF method to effectively capture the features of the dynamic complicated periods. FRNet decomposes the original time series into period and trend components. Based on the complex-valued linear networks, it leverages a period frequency rotation module to predict the period component and a patch frequency rotation module to predict the trend component, respectively. Extensive experiments on seven real-world datasets consistently demonstrate the superiority of FRNet over various state-of-the-art methods. The source code is available at https://github.com/SiriZhang45/FRNet.

The current virtual reality (VR) techniques develop immersive environments via inducing illusions to our sense. Nowadays, most of VR focuses on inducing visual illusion. Hence, visual is the most important input channel for experiencing and exploring the VR environments. Recently, extensive research efforts have been put on eye-tracking studies. However, the development and growing trends of the VR-based eye-tracking studies are unrevealed due to the lack of a systematic literature review on it. In this study, we reviewed related literature from 2000 to 2019 and summarized them into two main categories, including eye tracking methods and eye-tracking-enabled applications, such as tracking gaze points to manipulate the VR environment, measuring user states, and evaluating the usability of VR based on eye-tracking data. Based on the literature review, we can find that eye-tracking can assist in developing adaptive VR systems and enhance users experience. While comparing with 2D environments, immersive VR environment still requires more deep studies in eye-tracking.

The maritime industry is switching to new types of fuel such as Liquefied Natural Gas (LNG). On one hand, these kinds of fuel are more sustainable to the environment, on the other hand, training on handling such fuel safely and dealing with emergency situation is necessary. Videos and lecture-based learning is commonly used to deliver such knowledge to the maritime trainees. In recent years, the advances in Virtual Reality (VR) have brought new opportunities to such training. It provides an immersive while safe environment for training on certain operations that are extraordinary or dangerous in real life. It also allows the learners to practice the tasks repeatedly. The VR-based training is mostly used for improving technical skills, however, to guarantee a more efficient and better assessment of trainee's performance, nontechnical skills such as decision making, situation awareness, vigilance are needed to be assessed and trained as well. In this paper, a VR-based LNG evacuation training system is presented. The system provides two training scenarios for learning the evacuation procedure. A novel human factors evaluation based on the behavioral data captured by VR was proposed and integrated with the training, which includes both technical and non-technical skills assessment. An experiment with 14 subjects was conducted to validate the human factors evaluation and to get feedback towards the VR-based training.

Recently, network embedding has attracted extensive research interest. Most existing network embedding models are based on Euclidean spaces. However, Euclidean embedding models cannot effectively capture complex patterns, especially latent hierarchical structures underlying in real-world graphs. Consequently, hyperbolic representation models have been developed to preserve the hierarchical information. Nevertheless, existing hyperbolic models only capture the first-order proximity between nodes. To this end, we propose a new embedding model, named Node2LV, that learns the hyperbolic representations of nodes using squared Lorentzian distances. This yields three advantages. First, our model can effectively capture hierarchical structures that come from the network topology. Second, compared with the conventional hyperbolic embedding methods that use computationally expensive Riemannian gradients, it can be optimized in a more efficient way. Lastly, different from existing hyperbolic embedding models, Node2LV captures higher-order proximities. Specifically, we represent each node with two hyperbolic embeddings, and make the embeddings of related nodes close to each other. To preserve higher-order node proximity, we use a random walk strategy to generate local neighborhood context. We conduct extensive experiments on four different types of real-world networks. Empirical results demonstrate that Node2LV significantly outperforms various graph embedding baselines.

Nowadays, increasing attention has been drawn to hybrid 2D-3D visualization tools, while evaluating them with a convenient and objective tool has only been carried out in a small number of areas. In this study, a revised radar chart-based usability evaluation approach was proposed. The approach was adopted to evaluate the hybrid 2D-3D radar display in air traffic management. The holding stack in air traffic management is analyzed and simulated, two generic tasks are designed accordingly. The hybrid 2D-3D radar display settings are evaluated based on six indicators from eye-tracking and brain dynamics data, namely, the frequency of fixation, fixation mean duration, fixation time on an area of interest, emotion, workload, and stress. The results reveal that the hybrid 2D-3D radar display induces spatial memory loss and high workload, while requires a shorter fixation duration.

Objectively measuring and monitoring human mental states in a non-intrusive way is important in improving the context-awareness of smart objects. One of the suitable bio-signals in measuring human mental states is aye-tracking data, as visual is the first channel of information collection. In addition, eye-tracking data shows the process of human-system interactions. Traditionally, many studies have been conducted to investigate the correlations between eye-tracking data and human mental states. Recently, with advanced artificial intelligence algorithms, the spatial and temporal patterns of eye-tracking data can be deeply analyzed for detecting human mental states. This study aims to explore and review eye-tracking parameters and state-of-art methods for mental states assessments. The study reveals that both statistical methods and novel methods, such as machine learning and deep learning have been applied to process eye-tracking data. Besides, novel features extracted from eye-tracking data, such as gaze-bin and entropy have been used in assessing human mental states. This review is expected to provide references for eye-tracking data analysis.

Abstract Shunting of trains is a task that requires meticulous adherence to all steps to guarantee safety for everyone involved during and after the procedure. These steps are currently taught using classical teaching materials, such as printouts, videos and training by experienced supervisors. However, due to limited availability of locomotives, hours for training and manpower, training of shunting operation becomes challenging in real life. In this paper, we implemented a lifelike, collaborative virtual environment for shunting training including a novel human factors evaluation system for fatigue and stress monitoring. An experiment with 12 subjects and 3 trainers has been designed and carried out to validate the usage of VR-based shunting training. Positive feedback toward the VR-based training was obtained from the subjects and trainers.

Virtual Reality (VR) has been used for training aircraft pilots, maritime seafarers, operators, etc as it provides an immersive environment with realistic lifelike quality. We developed and implemented a VR-based Liquefied Natural Gas (LNG) firefighting simulation system with head-mounted displays (HMD) and novel human factors evaluation that could train and assess both technical and non-technical skills in the firefighting scenarios. The proposed human factors evaluation is based on a competence model and the non-technical skills such as situation awareness, vigilance, and decision making of seafarers could be assessed. An experiment was carried out with 6 trainees and 2 trainers using the implemented LNG firefighting simulation system. The results show that that the maritime trainees felt the VR scene was realistic to them, evoked similar emotions (such as fear, stress) during the demanding events as in the real world and made them attentive during the experience.

With the increasing popularity of location-aware social media services, next-Point-of-Interest (POI) recommendation has gained significant research interest. The key challenge of next-POI recommendation is to precisely learn users' sequential movements from sparse check-in data. To this end, various embedding methods have been proposed to learn the representations of check-in data in the Euclidean space. However, their ability to learn complex patterns, especially hierarchical structures, is limited by the dimensionality of the Euclidean space. To this end, we propose a new research direction that aims to learn the representations of check-in activities in a hyperbolic space, which yields two advantages. First, it can effectively capture the underlying hierarchical structures, which are implied by the power-law distributions of user movements. Second, it provides high representative strength and enables the check-in data to be effectively represented in a low-dimensional space. Specifically, to solve the next-POI recommendation task, we propose a novel hyperbolic metric embedding (HME) model, which projects the check-in data into a hyperbolic space. The HME jointly captures sequential transition, user preference, category and region information in a unified approach by learning embeddings in a shared hyperbolic space. To the best of our knowledge, this is the first study to explore a non-Euclidean embedding model for next-POI recommendation. We conduct extensive experiments on three check-in datasets to demonstrate the superiority of our hyperbolic embedding approach over the state-of-the-art next-POI recommendation algorithms. Moreover, we conduct experiments on another four online transaction datasets for next-item recommendation to further demonstrate the generality of our proposed model.

Eye-tracking-based human fatigue detection at traffic control centers suffers from an unavoidable problem of low-quality eye-tracking data caused by noisy and missing gaze points. In this article, the authors conducted pioneering work by investigating the effects of data quality on eye-tracking-based fatigue indicators and by proposing a hierarchical-based interpolation approach to extract the eye-tracking-based fatigue indicators from low-quality eye-tracking data. This approach adaptively classified the missing gaze points and hierarchically interpolated them based on the temporal-spatial characteristics of the gaze points. In addition, the definitions of applicable fixations and saccades for human fatigue detection is proposed. Two experiments are conducted to verify the effectiveness and efficiency of the method in extracting eye-tracking-based fatigue indicators and detecting human fatigue. The results indicate that most eye-tracking parameters are significantly affected by the quality of the eye-tracking data. In addition, the proposed approach can achieve much better performance than the classic velocity threshold identification algorithm (I-VT) and a state-of-the-art method (U'n'Eye) in parsing low-quality eye-tracking data. Specifically, the proposed method attained relatively stable eye-tracking-based fatigue indicators and reported the highest accuracy in human fatigue detection. These results are expected to facilitate the application of eye movement-based human fatigue detection in practice.

Currently, many modern humanoid robots have little appeal due to their simple designs and bland appearances. To provide recommendations for designers and improve the designs of humanoid robots, a study of human's perception on humanoid robot designs is conducted using Electroencephalogram (EEG), eye tracking information and questionnaires. We proposed and carried out an experiment with 20 subjects to collect the EEG and eye tracking data to study their reaction to different robot designs and the corresponding preference towards these designs. This study can possibly give us some insights on how people react to the aesthetic designs of different humanoid robot models and the important traits in a humanoid robot design, such as the perceived smartness and friendliness of the robots. Another point of interest is to investigate the most prominent feature of the robot, such as the head, facial features and the chest. The result shows that the head and facial features are the focus. It is also discovered that more attention is paid to the robots that appear to be more appealing. Lastly, it is affirmed that the first impressions of the robots generally do not change over time, which may imply that a good humanoid robot design impress the observers at first sight.

With the recent development of eye tracking technology, research in eye movement and pattern has increased due to its potential to be a non-obstructive physiological measure tool. This study attempts to understand to which extent the eye behavior is relatable with human's mental chronometry in responding to changes subjected to different levels of mental fatigue. An analysis of the eye movement metrics when interacting with multiple short performance-based tasks under different states of mental fatigue is performed. It is concluded that the eye movement has influence in the resulting reaction time and the mental fatigue state of the individual. Thus, indicating the relationship as a strong potential to predict an individual's mental fatigue state. Another finding is that the relationship between the eye movement metrics and mental chronometry becomes stronger as the subjective mental fatigue level increases.