Urban mobility is on the cusp of transformation with the emergence of shared, connected, and cooperative automated vehicles. Yet, for them to be accepted by customers, trust in their punctuality is vital. Many pilot initiatives operate without a fixed schedule, enhancing the importance of reliable arrival time (AT) predictions. This study presents an AT prediction system for automated shuttles, utilizing separate models for dwell and running time predictions, validated on real-world data from six cities. Alongside established methods such as XGBoost, we explore the benefits of leveraging spatial correlations using graph neural networks (GNN). To accurately handle the case of a shuttle bypassing a stop, we propose a hierarchical model combining a random forest classifier and a GNN. The results for the final AT prediction are promising, showing low errors even when predicting several stops ahead. Yet, no single model emerges as universally superior, and we provide insights into the characteristics of pilot sites that influence the model selection process and prediction performance. Finally, we identify dwell time prediction as the key determinant in overall AT prediction accuracy when automated shuttles are deployed in low-traffic areas or under regulatory speed limits. Our meta-analysis across six pilot sites in different cities provides insights into the current state of autonomous public transport prediction models and paves the way for more data-informed decision-making as the field advances.

Hierarchical policies enable strong performance in many sequential decision-making problems, such as those with high-dimensional action spaces, those requiring long-horizon planning, and settings with sparse rewards. However, learning hierarchical policies from static offline datasets presents a significant challenge. Crucially, actions taken by higher-level policies may not be directly observable within hierarchical controllers, and the offline dataset might have been generated using a different policy structure, hindering the use of standard offline learning algorithms. In this work, we propose $\textit{OHIO}$: a framework for offline reinforcement learning (RL) of hierarchical policies. Our framework leverages knowledge of the policy structure to solve the $\textit{inverse problem}$, recovering the unobservable high-level actions that likely generated the observed data under our hierarchical policy. This approach constructs a dataset suitable for off-the-shelf offline training. We demonstrate our framework on robotic and network optimization problems and show that it substantially outperforms end-to-end RL methods and improves robustness. We investigate a variety of instantiations of our framework, both in direct deployment of policies trained offline and when online fine-tuning is performed. Code and data are available at https://ohio-offline-hierarchical-rl.github.io.

Rapid urbanization in the past decades has significantly escalated mobility demand and imposed higher service quality standards. As a promising solution to address this challenge, Autonomous Mobility-on-Demand (AMoD) systems offer tailored mobility services while facilitating centralized control. In this paper, we formulate the joint rebalancing and dynamic pricing problem in AMoD systems as a reinforcement learning problem over graph elements. By proposing a hierarchical policy, we exploit the benefits of both optimization and reinforcement learning methods. Through experiments conducted with real-world data from New York City and San Francisco, we demonstrate that, by leveraging the joint policy, the two control mechanisms can work in tandem to effectively address the limitations inherent to independent dynamic pricing or rebalancing policies under different scenarios. Crucially, we show that our approach is effective when learning from both (1) online interaction with the transportation system and (2) offline from static historical data, thus avoiding the potentially expensive interaction needed for training. The success of our proposed framework in improving system performance under different problem settings underscores its potential as a viable solution for controlling real-world transportation systems.

The condition of lubricant oil is crucial for industrial machinery, as it minimizes wear and extends the equipment’s lifetime. A predictive maintenance approach that assesses oil condition in real time and forecasts the remaining useful lifetime can prevent costly component failures and machine downtime. The predictive maintenance concept presented here includes anomaly detection and real time oil condition estimation. Data is collected using an equivalent set of sensors at both a laboratory evaluation testbed and a production environment facility. A gold standard laboratory test provides information on the current oil condition. For anomaly detection, it is demonstrated that a Gaussian mixture model effectively identifies distribution shifts in temperature dependent sensor readings. For real time oil condition estimation, the phosphorus concentration is selected as the target value. A random forest regressor and a Gaussian process regressor are evaluated using a grouped cross-validation dataset. As an example, the predicted phosphorus concentration from the field data aligns closely with laboratory analysis results. To further validate these findings, additional field data from various use cases is needed. In the future, this data could help refine the model and incorporate an estimate of the remaining useful life.