User interface (UI) adaptation plays a crucial role in enhancing user experience by dynamically adjusting to individual preferences and situational needs. This study presents a comparative analysis of Model-Based Reinforcement Learning (MBRL) and Model-Free Reinforcement Learning (MFRL) as intelligent controllers of UI adaptations. We build upon an implementation of MFRL for emotion-driven UI adaptation and integrate an MBRL approach using Monte Carlo Tree Search (MCTS) with Upper Confidence Trees (UCT) for decision planning, along with a neural network for simulating user emotion responses. Experiments were conducted in a mobile EvacuationApp designed for emergency training, where participants interacted with adaptive UIs driven by either MBRL or MFRL. Our findings indicate that MBRL excels in scenarios with predictable user behavior, where a model of the interaction can be learned, whereas MFRL is more effective in dynamic, uncertain environments with frequent changes in user behavior. In our user study, MFRL achieved superior overall performance in eliciting the intended emotional responses, notably increasing users’ happiness level over successive adaptation cycles. This work offers a deep understanding of reinforcement learning-based UI adaptation strategies, examining the benefits and limitations of each approach.

Adaptive games increasingly combine player modeling, affect sensing, and runtime control, but evidence for how large language models (LLMs) and Reinforcement Learning (RL) could participate in such loops remains limited. We present an affect-aware game adaptation architecture that combines a warm-started tabular Q-learning controller with bounded LLM-assisted action suggestion inside a MAPE-K loop. The LLM is not allowed to act freely: it can only recommend one action from a fixed, predefined adaptation menu, while reinforcement learning remains responsible for value updates and policy improvement. We implemented the architecture in CubeWars, a top-down shooter instrumented with browser-side facial-expression sensing and four concrete adaptation levers: difficulty, zombie color, background audio, and modal gameplay prompts. We evaluated three within-subject conditions with 46 participants: a static baseline, RL-only adaptation, and LLM-assisted RL adaptation. The strongest objective result is that both adaptive conditions improved absolute progression outcomes relative to the static baseline, whereas RL-only and LLM-assisted RL did not differ significantly on normalized objective throughput measures. The clearest hybrid-specific benefits were experiential: the LLM-assisted condition produced higher engagement, stronger absorption, and greater awareness of adaptation than the RL-only condition. For affect, the hybrid condition showed the highest mean value on an exploratory reward-relevant emotion composite, but the pairwise hybrid-versus-RL-only contrast did not survive Bonferroni correction; we therefore interpret the affective findings primarily through their per-emotion decomposition and event-level analyses. The paper contributes: 1) a pattern for integrating constrained LLM advice into a self-adaptive interactive system without replacing the underlying RL controller; 2) an artifact-grounded implementation of affect-driven runtime game adaptation in a playable game; and 3) an empirical evaluation showing that, in the present evidence, the main added value of the hybrid design lies in perceived responsiveness and player experience.

The engineering of adaptive user interfaces has traditionally relied on either rule-based systems encoding designer intuitions about user needs or machine learning approaches requiring substantial historical data before achieving effective personalization. We present a technical architecture that leverages Large Language Models as behavioral synthesis engines to enable immediate adaptation from sparse, heterogeneous user signals. Our system integrates three distinct behavioral channels, i) explicit micro-feedback on individual interface elements, ii) spatial priority inferred from manual widget reorganization through drag-and-drop interaction, iii) and attentional investment measured through dwell time during hover events, within a structured prompt engineering framework that continuously regenerates dashboard layouts while maintaining explanatory coherence. The architecture addresses the technical challenge of translating low-level interaction patterns into high-level design decisions through a layered prompt construction methodology that separates temporal context determination, behavioral signal extraction, explicit preference enforcement, and user profile synthesis. The approach combines manually specified behavioral interpretations and temporal heuristics with LLM-mediated synthesis, enabling the reconciliation of multiple simultaneous signals that would be difficult to encode through explicit rules alone. We demonstrate the system through an instantiation in the personal health monitoring domain, including an analytical evaluation of adaptation behavior across multiple scenarios and a working implementation managing fourteen distinct health metrics across seven widget visualization modalities. The evaluation compares profile-driven initialization, multi-signal behavioral adaptation, and presents the resulting interfaces through representative post-adaptation screenshots. The analytical evaluation shows that the system preserves explicit user constraints, keeps user-prioritized metrics in prominent positions, expands widgets that receive sustained attention. The technical contribution comprises the multi-modal behavioral aggregation strategy, the structured LLM prompt engineering approach for maintaining design consistency across regeneration cycles, and the explainability generation mechanism that exposes adaptation rationale to end users. Our work provides a reproducible engineering approach for building LLM-powered adaptive interfaces that can be generalized beyond health dashboards to any domain requiring continuous interface personalization from heterogeneous user behavior. A current limitation is that, while the system can infer and act on behavioral signals, it does not yet incorporate mechanisms to independently verify whether adaptations improve user experience without additional feedback signals.

Understanding and designing Cyber-physical systems (CPS) with humans in the loop (HITL) is a basic cross-scientific research problem with large implications for industry. The current software engineering knowledge already explains how to include the humans in the operation of the machines in terms of interfaces, architectures, adaptive systems, and design methodologies for including the Human-in-the-Loop. This paper extends existing knowledge with a systematic review of socio-technical perspectives on CPS with HITL. The review was software engineering focused, as it searched the body of research on CPS with HITL, and only within that body, those papers that included socio-technical perspectives. The results indicated four main areas in the ST literature. Validating these insights by expert interviews with industry CPS experts showed some alignment and also fundamental differences between the socio-technical literature (ST literature) insights and the industry experts’ viewpoints. The discussion identifies useful crossings between the ST literature and research into CPS with HITL adaption, and touch on the issues of non-alignments in industry practice. The conclusion is that the ST perspectives on the body of knowledge on CPS with HITL has much to offer researchers in terms of innovative ways to look at the HITL, but the literature needs further development before industry experts can effectively use it. Future research possibilities are outlined.

Interactive computer systems’ designers emphasize the importance of considering humans, their emotions, and behaviors as first-class entities. Emotions are integral parts of human nature, and ignoring that can lead the interactive systems to failure, low quality, or discomfort. User interfaces (UIs) are increasingly becoming adaptive to users’ various characteristics, intending to improve users’ satisfaction, performance, and decisions. However, the previous approaches proposed for supervising such adaptations are not effectively adopted in real-life problems. This paper proposes the novel approach to adapting UIs to users’ emotions using Model-Free Reinforcement Learning (MFRL). The approach aims to maximize applying the essential adaptations and minimize the unnecessary ones towards users’ task completion and satisfaction. We chose emergency evacuation training as a suitable evaluation domain since people experience intense emotions in potential danger. We performed experiments with a mobile application we developed that acts as a recommender system in emergency training. By taking contextual input of the users’ basic emotions from face recognition, the application intelligently adapts its UI to quickly lead people to safe areas while arousing target emotions. The research includes literature analysis, surveys, and further adopting an iterative process in implementation and experimentation. The evaluation process confirms the efficiency and effectiveness of the MFRL in iterations, as well as compared to other possible UI adaptation techniques, i.e., rule-based and sequential adaptation.

Emotions are integral to human nature, and their existence, duration, and evolution could lead to specific behaviors. If emotions and behaviors are ignored in the design of socio-technical systems, they will fail or cause discomfort. User interfaces (UIs) are elements of interactive systems able to trigger or moderate emotions. UIs are increasingly designed adaptive to users' various characteristics, intending to improve their satisfaction, performance, and decisions. However, previous adaptation supervising approaches are not effectively adopted in real life since they neglect the dynamic behaviors of humans or systems. This paper proposes Emoticontrol, a quality-driven approach to adapting UIs to users' emotions using Model-Free Reinforcement Learning (MFRL). The approach aims to maximize applying the essential adaptations and minimize the unnecessary ones towards users' enhanced quality of experience (QoE). The approach also considers improving the software quality of service (QoS) by designing software architecture alternatives. We chose emergency evacuation training as a suitable evaluation domain since people experience intense emotions in potential danger. We performed experiments with a mobile application we developed that acts as a recommender system in evacuation training. By taking contextual input of the users' basic emotions from face recognition, the application intelligently adapts its UI to quickly lead people to safe areas while keeping them emotionally controlled. We consider software performance a crucial QoS; thus, we adopt and test architectures that facilitate an acceptable level of performance. The evaluation process confirms the efficiency and effectiveness of the MFRL in iterations, as well as compared to other UI adaptation techniques.

The Internet of Behaviors (IoB) puts human behavior at the core of engineering intelligent connected systems. IoB links the digital world to human behavior to integrate human-driven design, development, and adaptation processes. This paper defines the novel IoB concept with a constructed model based on a collective effort interacting with software engineers, human-computer interaction scientists, social scientists, and cognitive science communities. The model for IoB is created based on an exploratory study that synthesizes state-of-the-art analysis and experts interviews. The architecture of a real industry 4.0 manufacturing infrastructure helps to explain the IoB model and its application. The conceptual model was used to successfully implement a socio-technical infrastructure for a crowd monitoring and queue management system for the Uffizi Galleries, Florence, Italy. The experiment, which started in the fall of 2016 and was operational in the fall of 2018, used a data-driven approach to feed the system with real-time sensory data. It also incorporated prediction models on visitors’ mobility behavior. The system’s main objective was to capture human behavior, model it, and build a mechanism that considers changes, adapts in real-time, and continuously learns from repetitive behaviors. In addition to the conceptual model and the real-life evaluation, this paper provides recommendations from experts and gives future directions for IoB to become a significant technological advancement in the coming years.