This work focuses on enhancing autonomous vehicle navigation systems by leveraging the semantic information contained in traffic signs, with a particular emphasis on text-based signs. We introduce an approach to interpret the text on signs for downstream autonomous vehicle applications, such as determining parking restrictions and identifying roadwork zones and directional commands. Specifically, we build upon our previously developed method [1] to create a more fine-grained representation of traffic sign parsing and utilize a pre-trained language model in a constrained setting. This approach helps alleviate the need for hand-crafted rules, making our method significantly more scalable. Our approach decomposes the parsing task into simpler tasks, each of which can be implemented using a single lightweight transformer-based language model, thereby improving efficiency and interpretability. We demonstrate the effectiveness of our method through experiments, showing substantial improvements in accuracy over a previously introduced baseline. To the best of our knowledge, this is the first application of a unified language model to this task, representing a significant step toward open-world sign interpretation.

Autonomous vehicles (AVs) are becoming increasingly prevalent. However, current AVs are unable to handle unexpected traffic signs (e.g., construction zones, road closures) encountered on the roads. To address this limitation, we propose MOSER, a Multimodal rOad Sign intERpretation system, to enable automated detection and interpretation of diverse road signs. Our system consists of a pipeline architecture with three main components, including perception, text processing, and planning. The perception component detects arbitrary road signs and extracts the sign text into proper groups and orders. The text processing component then identifies the high-level semantics of the text and determines whether any actions are required for the autonomous vehicle. Based on the interpretation of the signs, the planning component provides navigation guidance, such as instructing the vehicle to stop at a specific location or adding rules to its internal map. To the best of our knowledge, this is the first attempt to address the interpretation of arbitrary road signs using a multimodal processing strategy. Our work provides important insights and capabilities to support Level 4 autonomous vehicles, ensuring their safety and smoothness of operation.

Concept maps are concise graphical representations of text data which have been shown to be applicable as a tool for text summarization and downstream tasks. Most prior work has either focused on the generation of concept maps for small corpora or require expensive training data to implement. In this work, we focus on generating causal maps, a subset of concept maps in which only semantically causal relationships are considered. We propose a map generation framework which utilizes a novel mixture model to simultaneously derive concepts and links. This method is computationally efficient and therefore scalable to large datasets, and is fully unsupervised, which makes it suitable for practical applications. We show that our method performs better than a commonly used unsupervised text summarization algorithm, and has results which are comparable to the state-of-the-art supervised method.