The mean intersection over union (mloU) is a standard metric for evaluating semantic segmentation models. While steady improvements in mloU have been achieved on automotive benchmarks like Cityscapes, their impact on reliably detecting vulnerable road users, such as pedestrians and riders, remains unclear. This study empirically analyzes 167 semantic segmentation models w.r.t. the spatial distribution of the false positive rate and false negative rate in the Cityscapes dataset. Our analysis reveals that many segmentation errors occur at object contours, which hardly influence driving decisions and road user safety. Accordingly, we propose to exclude such irrelevant errors. We define spatial error bounds within which models reliably detect pedestrians and riders. Since time-to-collision is strongly related to distance, and a vertical pixel position is roughly related to distance, the vertical position of segmentation errors provides an effective way to evaluate the reliability of semantic segmentation models on an entire dataset. Our evaluation of such empirical spatial error bounds reveals that strong models (w.r.t. mloU) are related to an improved detection of existing pedestrians (false negative rate, FNR). On the other hand, mloU in general is only weakly related to hallucinations of pedestrians and riders (false positive rate, FPR). Some models even exhibit a higher FPR despite having a 11.2% absolute higher mloU.

Many approaches in autonomous driving generate future waypoints to form trajectories, which are then used to derive driving commands. During imitation learning for end-to-end autonomous driving, these trajectories are typically learned using a straightforward$L_{1}$loss, which compares the model's predictions to those of an expert. In this paper, we propose a separation of longitudinal and lateral components of the$L_{1}$loss that weighs these independently, thereby aligning more with the separate handling of longitudinal and lateral control by PID controllers in the model pipeline. We employ this novel generalized waypoint loss with the TransFuser architecture in the CARLA simulator and show that we can control and improve on certain infraction types, without a performance loss in any other metric. Additionally, we investigate a novel ensemble technique that produces a more cautious ensemble, reducing infractions while maintaining overall performance. For future work, our novel loss formulation enables the definition of a time-variant loss tailored to specific traffic scenarios in the training data.

While Multimodal Large Language Models (MLLMs) offer strong perception and reasoning capabilities for image-text input, Visual Question Answering (VQA) focusing on small image details still remains a challenge. Although visual cropping techniques seem promising, recent approaches have several limitations: the need for task-specific fine-tuning, low efficiency due to uninformed exhaustive search, or incompatibility with efficient attention implementations. We address these shortcomings by proposing a training-free visual cropping method, dubbed FOCUS, that leverages MLLM-internal representations to guide the search for the most relevant image region. This is accomplished in four steps: first, we identify the target object(s) in the VQA prompt; second, we compute an object relevance map using the key-value (KV) cache; third, we propose and rank relevant image regions based on the map; and finally, we perform the fine-grained VQA task using the top-ranked region. As a result of this informed search strategy, FOCUS achieves strong performance across four fine-grained VQA datasets and three types of MLLMs. It outperforms three popular visual cropping methods in both accuracy and efficiency, and matches the best-performing baseline, ZoomEye, while requiring 3 – 6.5 × less compute.