Fault detection and isolation are becoming increasingly important as modern systems become more complex. To encourage the development of new fault detection solutions that can operate with limited noisy data and an incomplete mathematical model, the DX 2025 LiU-ICE competition for diagnosis of the air path of an internal combustion engine was introduced. In this paper, we present our winning solution to this competition. Our fault detection architecture starts with a semi-supervised Transformer Autoencoder trained to reconstruct nominal data. Detected faults are then passed through a rule-based fault persistence filter that aims to suppress false positives. Once a fault is detected, we use four neural networks trained to estimate features determined from structural analysis of a partial system model. The residuals of these networks are fed to a supervised fault classification network that estimates the fault probabilities. With this architecture, we achieved an 87% detection rate with a 0% false alarm rate on the provided competition data. Additionally, our isolation architecture assigned the correct fault 73.8% probabilty on average. On unseen competition data from a new driving cycle, we achieved a 100% detection rate and assigned the correct fault 66.2% probability on average. On the other hand, the Transformer Autoencoder failed to transfer to the new driving conditions, causing many false alarms. We discuss ways future work can reduce this.

Accurate estimation of the remaining useful life (RUL) of industrial systems is a critical component of predictive maintenance strategies. This work presents a data-driven method for RUL prediction that also quantifies uncertainty, drawing inspiration from model-based particle filtering techniques. Instead of simulating system state transitions, we model degradation as a stochastic process governed by performance metrics and use a Bayesian particle filtering framework to infer its underlying parameters. Our approach bypasses traditional state-space modeling by directly estimating the end-of-life distribution from observed performance data. Key characteristics of the filter, such as propagation noise and observation correction strength, are adapted over time based on current observations and past predictive performance, enabling better capture of future uncertainty. We evaluate the proposed method using an unmanned aerial vehicle simulation dataset developed for system-level prognostics research, which includes high-fidelity degradation signals and ground-truth system performance metrics for validating predictive accuracy.

Fully-automated optimal controller design for engineering systems is a challenging task. While, optimization-based, automated control parameter tuning techniques have been widely discussed in the literature, most works do not discuss expert knowledge requirements for system design, which result in significant human intervention. In this work, we discuss a multistage controller tuning framework for decentralized control that highlights expert knowledge requirements in automated controller design. We propose a methodology to automate the input-output pairing and stage definition steps in the framework using Large Language Models (LLMs) for a family of multi-tank benchmarks. We achieve this by proposing a mathematical language to describe the system and design an algorithm to bind this mathematical representation to the input prompt space of an LLM. We demonstrate that our methodology can produce consistent expert knowledge outputs from the LLM with over 97% accuracy for the multi-tank benchmarks. We also empirically show that, correct stage definition by the LLM can improve tuned controller performance by up to 52%.

Ensuring flight safety for small unmanned aerial systems (sUAS) requires continuous in-flight monitoring and decision-making, as unexpected events can alter power consumption and deplete battery energy faster than anticipated. Such events may result in insufficient battery capacity to complete a mission, thereby compromising flight safety. In this paper, we present an online feasibility assessment and contingency management framework that continuously monitors the aircraft’s battery state and the energy required to complete the flight in real-time, which enables informed decision-making to enhance flight safety. The framework consists of two main components: power consumption prediction and battery voltage trajectory prediction. The power consumption prediction is conducted using a model that is based on momentum theory, while the voltage trajectory prediction is performed using a Neural Ordinary Differential Equation (Neural ODE)-based data-driven model. By integrating these two components, the framework evaluates the feasibility of a flight mission in real time and determines whether to proceed with the mission or initiate rerouting. We evaluate the framework’s performance in a drone delivery scenario in the Dallas–Fort Worth (DFW) area, where the aircraft encounters an unexpected energy depletion event mid-flight. The proposed framework is tasked with assessing the feasibility of completing the mission and, if necessary, rerouting the aircraft for an emergency landing. The results demonstrate that the framework accurately and efficiently detects energy insufficiencies in real-time and re-routes the aircraft to a [3] predefined emergency landing site.

Offline reinforcement learning (RL) enables learning decision-making policies directly from historical data, which is advantageous for safety-critical domains like traffic control. However, existing offline RL benchmarks in transportation systems typically rely on simulated data, which may not fully capture the complexities of real-world environments. In this paper, we introduce the first offline RL benchmark for variable speed limit (VSL) control, built from approximately 100 million transitions of real-world interaction data collected from a field-deployed, multi-agent RL-based VSL system on a major freeway. We evaluate five state-of-the-art offline RL algorithms under multiple dataset conditions defined by varying sizes and action noise levels. Through traffic microsimulation experiments, we analyze algorithm performance and generalization, providing insights into the challenges and opportunities of offline RL for intelligent transportation systems. The dataset and benchmark are released at https://github.com/Lab-Work/i24-vsl-orl.

Fully-automated optimal design of Cyber-Physical Systems is a challenging task that involves multiple components. In this paper, we are interested in the automated design of one of these components: the system controller. State-of-the-art con-trol parameter automated tuning techniques are optimization-based. However, for high-dimension control parameter spaces, computational costs can be high. We present a multistage controller tuning framework that decomposes controller tuning into sub-tasks, each with a reduced-dimension search space. We show formally that this framework reduces the sample complexity of the control-tuning task. We empirically validate this result by applying a Bayesian optimization approach to tuning multiple PID controllers in an unmanned underwater vehicle benchmark system. We demonstrate an 86 % decrease in computational time and a 36 % decrease in sample complex-ity. Furthermore, the proposed framework highlights existing challenges in fully automated control parameter tuning.

Due to the safety risks and training sample inefficiency, it is often preferred to develop controllers in simulation. However, minor differences between the simulation and the real world can cause a significant sim-to-real gap. This gap can reduce the effectiveness of the developed controller. In this paper, we examine a case study of transferring an octorotor reinforcement learning controller from simulation to the real world. First, we quantify the effectiveness of the real-world transfer by examining safety metrics. We find that although there is a noticeable (around 100%) increase in deviation in real flights, this deviation may not be considered unsafe, as it will be within > 2m safety corridors. Then, we estimate the densities of the measurement distributions and compare the Jensen-Shannon divergences of simulated and real measurements. From this, we show that the vehicle’s orientation is significantly different between simulated and real flights. We attribute this to a different flight mode in real flights where the vehicle turns to face the next waypoint. We also find that the reinforcement learning controller actions appear to correctly counteract disturbance forces. Then, we analyze the errors of a measurement autoencoder and state transition model neural network applied to real data. We find that these models further reinforce the difference between the simulated and real attitude control, showing the errors directly on the flight paths. Finally, we discuss important lessons learned in the sim-to-real transfer of our controller.

Early and accurate detection of anomalous events on the freeway, such as accidents, can improve emergency response and clearance. However, existing delays and mistakes from manual crash reporting records make it a difficult problem to solve. Current large-scale freeway traffic datasets are not designed for anomaly detection and ignore these challenges. In this paper, we introduce the first large-scale lane-level freeway traffic dataset for anomaly detection. Our dataset consists of a month of weekday radar detection sensor data collected in 4 lanes along an 18-mile stretch of Interstate 24 heading toward Nashville, TN, comprising over 3.7 million sensor measurements. We also collect official crash reports from the Tennessee Department of Transportation Traffic Management Center and manually label all other potential anomalies in the dataset. To show the potential for our dataset to be used in future machine learning and traffic research, we benchmark numerous deep learning anomaly detection models on our dataset. We find that unsupervised graph neural network autoencoders are a promising solution for this problem and that ignoring spatial relationships leads to decreased performance. We demonstrate that our methods can reduce reporting delays by over 10 minutes on average while detecting 75% of crashes. Our dataset and all preprocessing code needed to get started are publicly released at https://vu.edu/ft-aed/ to facilitate future research.

In this paper, we leverage ideas from model-based control to address the sample efficiency problem of reinforcement learning (RL) algorithms. Accelerating learning is an active field of RL highly relevant in the context of time-varying systems. Traditional transfer learning methods propose to use prior knowledge of the system behavior to devise a gradual or immediate data-driven transformation of the control policy obtained through RL. Such transformation is usually computed by estimating the performance of previous control policies based on measurements recently collected from the system. However, such retrospective measures have debatable utility with no guarantees of positive transfer in most cases. Instead, we propose a model-based transformation, such that when actions from a control policy are applied to the target system, a positive transfer is achieved. The transformation can be used as an initialization for the reinforcement learning process to converge to a new optimum. We validate the performance of our approach through four benchmark examples. We demonstrate that our approach is more sample-efficient than fine-tuning with reinforcement learning alone and achieves comparable performance to linear-quadratic-regulators and model-predictive control when an accurate linear model is known in the three cases. If an accurate model is not known, we empirically show that the proposed approach still guarantees positive transfer with jump-start improvement.

The optimal energy performance of building operations is affected by component faults, which may go unnoticed for long periods of time. Significant energy savings can be achieved if faulty behaviors are detected and rectified in a timely manner. In this work, we adopt an unsupervised approach for anomaly detection that combines automatic data engineering using clustering methods with Long-Short Term Memory (LSTM)-based Autoencoders. First, data engineering is used to extract multiple operating modes from nominal data of building operations. Then, an LSTM-based Autoencoder is trained to capture the characteristics of non-linear and temporal dynamics for each operating mode. Finally, the ensemble of models can be used for anomaly detection after training has been completed. We benchmark variants of our approach against state-of-the-art Autoencoders for anomaly detection by using a recently developed experimental dataset provided by the ASHRAE Research Project RP-1312. Unsupervised anomaly detection is a challenging task due to the lack of faulty labels and the need to identify faults while avoiding false alarms. Our novel approach improves the average true positive rate for fault detection by 11.4% against a state-of-the-art plain LSTM Autoencoder while keeping the false alarm rate around 5 % without having to use labeled fault data.

Accurate estimation of the battery state of charge (SOC) for unmanned aerial vehicles (UAV) in-flight monitoring is essential for the safety and survivability of the system. Successful physics-based models of the battery have been developed in the past, however, these models do not take into account the effects of mission profile and environmental conditions during flight on the battery power consumption. Recently, data-driven methods have become popular given their ease of use and scalability. Yet, most benchmarking experiments have been conducted on simulated battery datasets. In this work, we compare different data-driven models for battery SOC estimation of a hexacopter UAV system using real flight data. We analyze the importance of a number of flight variables under different environmental conditions to determine the factors that affect battery SOC over the course of the flight. Our experiments demonstrate that additional flight variables are necessary to create an accurate SOC estimation model through data-driven methods.

Variable speed limit (VSL) control has emerged as a promising traffic management strategy for enhancing safety and mobility. In this study, we introduce a multi-agent reinforcement learning framework for implementing a large-scale VSL system to address recurring congestion in transportation corridors. The VSL control problem is modeled as a Markov game, using only data widely available on freeways. By employing parameter sharing among all VSL agents, the proposed algorithm can efficiently scale to cover extensive corridors. The agents are trained using a reward structure that incorporates adaptability, safety, mobility, and penalty terms; enabling agents to learn a coordinated policy that effectively reduces spatial speed variations while minimizing the impact on mobility. Our findings reveal that the proposed algorithm leads to a significant reduction in speed variation, which holds the potential to reduce incidents. Furthermore, the proposed approach performs satisfactorily under varying traffic demand and compliance rates.

In this work, we address the problem of solving complex collaborative robotic tasks subject to multiple varying parameters. Our approach combines simultaneous policy blending with system identification to create generalized policies that are robust to changes in system parameters. We employ a blending network whose state space relies solely on parameter estimates from a system identification technique. As a result, this blending network learns how to handle parameter changes instead of trying to learn how to solve the task for a generalized parameter set simultaneously. We demonstrate our scheme's ability on a collaborative robot and human itching task in which the human has motor impairments. We then showcase our approach's efficiency with a variety of system identification techniques when compared to standard domain randomization. The code is available on Luke Bhan's Github.

The control of complex autonomous systems has significantly improved in recent years and unmanned aerial vehicles (UAVs) have become popular in the research community. Although the use of UAVs is increasing, much work remains to guarantee fault- tolerant control (FTC) properties of these vehicles. Model-based controllers are the standard way to control UAVs, however obtaining models of the system and environment for every possible operating condition a UAV can experience in a real-world scenario is not feasible. Reinforcement Learning has shown promise in controlling complex systems but requires training in a simulator (requiring a model) of the system. Further, stability guarantees do not exist for learning-based controllers, which limits their large scale application in the real-world. We propose a novel hybrid FTC approach that uses a learned supervisory controller (together with low-level PID controllers) with key stability guarantees. We use a robust reinforcement learning approach to learn the supervisory control parameters and prove stability. We empirically validate our framework using trajectory-following experiments (in simulation) for a quadcopter subject to rotor faults, wind disturbances, and severe position and attitude noise.