Offline Reinforcement Learning (RL) aims to learn policies by using static datasets. The extrapolation error in out-of-distribution (OOD) samples can cause off-policy RL algorithms to perform poorly on offline datasets. Hence, it is critical to avoid visiting OOD states and taking OOD actions in offline RL. Several recent methods have used uncertainty estimation to distinguish OOD samples. However, errors in the uncertainty estimation make the purely uncertainty-based method unstable and require additional components to ensure sufficient pessimism . In this study, we propose a Bounded Uncertainty based Pessimistic policy iteration algorithm (BUP). The BUP pessimistically estimates the value function via bounded uncertainty, and the uncertainty bound is achieved by constraining the actor from taking highly uncertain actions. The suboptimality bound of BUP is theoretically guaranteed in linear Markov Decision Processes (MDPs), and experiments on D4RL datasets show that BUP matches the state-of-the-art performance. Moreover, BUP is simple to implement with low computational cost and does not require any additional components.

Pedestrian crossing intention prediction is crucial for autonomous vehicles due to their inherent inertia, yet challenging. The prevailing practice is to leverage multi-modal data that correlate with pedestrian crossing intention as input to infer it, with ego-vehicle speed being a commonly used modality. However, from a causal perspective, we identify two critical issues overlooked by existing methods: 1) The causal relationship between ego-vehicle speed and pedestrian crossing intention is inconsistent across the training and testing phases, leading to a distribution shift in the data; 2) There exists a bidirectional causality between ego-vehicle speed and pedestrian crossing intention, comprising forward causality and anti-causality. The imbalanced distribution of these two causal directions in natural datasets results in causal confusion, further exacerbating the distribution shift. These issues lead to a counter-intuitive hypothesis: removing ego-vehicle speed as input can actually benefit prediction performance. Therefore, we propose LIM (Less Is More), a uni-modal model that utilizes only skeleton sequences as input. LIM features specially designed modules for efficient skeleton sequence processing, eliminating the reliance on unstable ego-vehicle speed. Moreover, LIM employs adversarial training to identify and remove any latent ego-vehicle speed information embedded within the skeleton sequences. LIM achieves competitive prediction performance compared to state-of-the-art multi-modal models while offering superior real-time capabilities and lower computational costs. By revealing critical issues overlooked by most existing work and providing a more robust crossing intention prediction model, our work contributes to the development of safer autonomous vehicles.

Offline reinforcement learning (RL) aims to learn policy from the passively collected offline dataset. Applying existing RL methods on the static dataset straightforwardly will raise distribution shift, causing these unconstrained RL methods to fail. To cope with the distribution shift problem, a common practice in offline RL is to constrain the policy explicitly or implicitly close to behavioral policy. However, the available dataset usually contains sub-optimal or inferior actions, constraining the policy near all these actions will make the policy inevitably learn inferior behaviors, limiting the performance of the algorithm. Based on this observation, we propose a weighted policy constraints (wPC) method that only constrains the learned policy to desirable behaviors, making room for policy improvement on other parts. Our algorithm outperforms existing state-of-the-art offline RL algorithms on the D4RL offline gym datasets. Moreover, the proposed algorithm is simple to implement with few hyper-parameters, making the proposed wPC algorithm a robust offline RL method with low computational complexity.