Large language models are increasingly influencing human moral decisions, yet current approaches focus primarily on evaluating rather than actively steering their moral decisions. We formulate this as an out-of-distribution moral alignment problem, where LLM agents must learn to apply consistent moral reasoning frameworks to scenarios beyond their training distribution. We introduce Moral-Reason-QA, a novel dataset extending 680 human-annotated, high-ambiguity moral scenarios with framework-specific reasoning traces across utilitarian, deontological, and virtue ethics, enabling systematic evaluation of moral generalization in realistic decision contexts. Our learning approach employs Group Relative Policy Optimization with composite rewards that simultaneously optimize decision alignment and framework-specific reasoning processes to facilitate learning of the underlying moral frameworks. Experimental results demonstrate successful generalization to unseen moral scenarios, with softmax-normalized alignment scores improving by +0.757 for utilitarian and +0.450 for deontological frameworks when tested on out-of-distribution evaluation sets. The experiments also reveal training challenges and promising directions that inform future research. These findings establish that LLM agents can be systematically trained to internalize and apply specific moral frameworks to novel situations, providing a critical foundation for AI safety as language models become more integrated into human decision-making processes.

Model-based reinforcement learning (MBRL) is widely studied for heating, ventilation, and air conditioning (HVAC) control in buildings. One of the critical challenges is the large amount of data required to effectively train neural networks for modeling building dynamics. This article presents CLUE, an MBRL system for HVAC control in buildings. CLUE optimizes HVAC operations by integrating a Gaussian process (GP) model to model building dynamics with uncertainty awareness. CLUE utilizes GP to predict state transitions as Gaussian distributions, effectively capturing prediction uncertainty and enhancing decision-making under sparse data conditions. Our approach employs a meta-kernel learning technique to efficiently set GP kernel hyperparameters using domain knowledge from diverse buildings. This drastically reduces the data requirements typically associated with GP models in HVAC applications. Additionally, CLUE incorporates these uncertainty estimates into a model predictive path integral (MPPI) algorithm, enabling the selection of safe, energy-efficient control actions. This uncertainty-aware control strategy evaluates and selects action trajectories based on their predicted impact on energy consumption and human comfort, optimizing operations even under uncertain conditions. Extensive simulations in a five-zone office building demonstrate that CLUE reduces the required training data from hundreds of days to just seven while maintaining robust control performance. It reduces comfort violations by an average of 12.07% compared to existing MBRL methods, without compromising on energy efficiency. Our code and dataset are available athttps://github.com/ryeii/CLUE.

Recent research has shown the potential of Model-based Reinforcement Learning (MBRL) to enhance energy efficiency of Heating, Ventilation, and Air Conditioning (HVAC) systems. However, existing methods rely on black-box thermal dynamics models and stochastic optimizers, lacking reliability guarantees and posing risks to occupant health. In this work, we overcome the reliability bottleneck by redesigning HVAC controllers using decision trees extracted from existing thermal dynamics models and historical data. Our decision tree-based policies are deterministic, verifiable, interpretable, and more energy-efficient than current MBRL methods. First, we introduce a novel verification criterion for RL agents in HVAC control based on domain knowledge. Second, we develop a policy extraction procedure that produces a verifiable decision tree policy. We found that the high dimensionality of the thermal dynamics model input hinders the efficiency of policy extraction. To tackle the dimensionality challenge, we leverage importance sampling conditioned on historical data distributions, significantly improving policy extraction efficiency. Lastly, we present an offline verification algorithm that guarantees the reliability of a control policy. Extensive experiments show that our method saves 68.4% more energy and increases human comfort gain by 14.8% compared to the state-of-the-art method, in addition to an 1127× reduction in computation overhead. Our code and data are available at https://github.com/ryeii/Veri_HVAC.

The rapid decline in groundwater around the world poses a significant challenge to sustainable agriculture. To address this issue, agricultural managed aquifer recharge (Ag-MAR) is proposed to recharge the aquifer by artificially flooding agricultural lands using surface water. Ag-MAR requires a carefully selected flooding schedule to avoid affecting the oxygen absorption of crop roots. However, current Ag-MAR scheduling does not take into account complex environmental factors such as weather and soil oxygen, resulting in crop damage and insufficient recharging amounts. This paper proposes MARLP, the first end-to-end data-driven control system for Ag-MAR. We first formulate Ag-MAR as an optimization problem. To that end, we analyze four-year in-field datasets, which reveal the multi-periodicity feature of the soil oxygen level trends and the opportunity to use external weather forecasts and flooding proposals as exogenous clues for soil oxygen prediction. Then, we design a two-stage forecasting framework. In the first stage, it extracts both the cross-variate dependency and the periodic patterns from historical data to conduct preliminary forecasting. In the second stage, it uses weather-soil and flooding-soil causality to facilitate an accurate prediction of soil oxygen levels. Finally, we conduct model predictive control (MPC) for Ag-MAR flooding. To address the challenge of large action spaces, we devise a heuristic planning module to reduce the number of flooding proposals to enable the search for optimal solutions. Real-world experiments show that MARLP reduces the oxygen deficit ratio by 86.8% while improving the recharging amount in unit time by 35.8%, compared with the previous four years.

Low-field (LF) nuclear magnetic resonance (NMR) technology has been widely used in reservoir identification, NMR logging and other geophysical exploration fields. Due to the advantage of providing abundant parameter information related to fluid (e.g. the longitudinal and transverse relaxation time,T1andT2), it requires the advanced inversion approach for data interpretation from the measured echo data. As the present inversion schemes show weakness, especially in fluid typing and quantitative analysis, we developed a novel imaging scheme based on the classical Butler–Reeds–Dawson (BRD) algorithm frame. Employing the discrepancy principle, the new method aims to improve the selection for regularized parameter for inversion, and further increases the imaging accuracy. To verify the ability and reliability of this method with strong interference, we compare the fluid typing inversion results of simulated data for different noise level between the conventional and improved approach with variable data acquisition wait time and echo spacing. The results indicate that the estimation accuracy ofT1-T2spectra for updated scheme gets about 8 percent improvement even with high-noise environments. To further evaluate the advantage, this algorithm is also tested with real oil-water fluid experimental data, which yields a distribution of fluid properties that matches the sample. In conclusion, the research in this paper shows significant for LF-NMR data interpretation, in particular for charactering pore-fluid and quantifying organic contamination in subsurface sediments in the geological survey.

Model-based Reinforcement Learning (MBRL) has been widely studied for energy-efficient control of the Heating, Ventilation, and Air Conditioning (HVAC) systems. One of the fundamental issues of the current approaches is the large amount of data required to train an accurate building system dynamics model. In this work, we developed a data-efficient system capable of excellent HVAC control performance with only days of training data. We use a Gaussian Process (GP) as the dynamics model which provides uncertainty for each prediction. To improve the data efficiency, we designed a meta kernel learning technique for GP kernel selection. To incorporate uncertainty in the control decisions, we designed a model predictive control method that considers the uncertainty of every prediction. Simulation experiments show that our method achieves excellent data efficiency, yielding similar energy savings and 12.07% less human comfort violation compared with the state-of-the-art MBRL method, while only trained on a seven-day training dataset.