Large Language Models (LLMs) have recently begun to excel at complex domain-specific questions thanks to techniques like Retrieval Augmented Generation (RAG). However, the spatiotemporal reasoning abilities of LLMs are still lacking. Spatial reasoning is required to answer questions in a variety of domains that are grounded in the physical world, including urban planning, civil engineering, travel, and others. To improve LLMs and facilitate an impact in these domains, we aim to enable LLMs to reason over spatial data. In this paper we outline the challenges associated with spatial reasoning through LLMs and describe a framework to integrate spatial data with LLMs which will allow them to answer complex spatial questions.

Traditional search engines use text-based queries to perform top-k keyword search. However, this approach does not always capture every user intention. For example, pattern-based spatial search can better answer queries involving spatial constraints (i.e. X North of Y). However, pattern-based search usually requires a pictorial query pattern as input, constructed by a user dragging and dropping objects on a canvas in a specialized interface. To bridge the gap between pattern-based spatial search and traditional search engines that require text input, we devise a Natural Language to Pictorial Query (NL2PQ) module that converts natural language queries into pictorial queries that can be refined then resolved using spatial pattern matching algorithms, thus enabling pattern based spatial search via natural language input.

Sun glare during driving poses a significant threat to driver and pedestrian safety. Navigation and route planning typically seeks to minimize the distance or time between the desired origin and destination, accounting for traffic patterns and other heuristics like minimizing the number of stoplights or left turns encountered on a route. However, current navigation methods do not support avoidance of complicated, temporally-dependent safety factors, like adverse road and environmental conditions. We take avoiding incident sun glare to the driver as an example of dynamic safety-aware navigation and lay out potential strategies for addressing this previously unexplored problem. We present a reinforcement learning-based method for computing sun glare-low routes through an elastic function that accounts for the direct angle between the sun and the driving direction. Our preliminary work shows that in some cases it is possible to reduce the sun glare exposure on a route by trading off additional travel distance. We envision future safety-aware navigation approaches that can automatically balance this trade-off and account for additional dynamic spatially and temporally-dependent safety-related environmental factors, like road and weather conditions, to determine the safest and most efficient route between any two given points.

Spatial reasoning is a particularly challenging task that requires inferring implicit information about objects based on their relative positions in space. In an effort to develop general purpose geo-foundation models that can perform a variety of spatial reasoning tasks, preliminary work has explored what kinds of world knowledge and spatial reasoning capabilities Large Language Models (LLMs) naturally inherit from their training data. Recent work suggests that LLMs contain geospatial knowledge in the form of understanding geo-coordinates and associating spatial meaning to the key terms "near" and "far." In this paper, we show that LLMs lack the ability to adapt the meaning of the words "near" and "far" to the appropriate scale when provided contextual reference points. By uncovering biases in how LLMs answer distance-related spatial questions, we set the groundwork for developing new techniques that may enable LLMs to perform accurate spatial reasoning.

Spatial Pattern Matching is an important search problem that involves reasoning about the relative position, distance, and orientation of objects with respect to one another. Spatial relationships between objects contain a lot of information about the world, which makes them useful in applications like Point of Interest (POI) retrieval and location-based services. However, spatial pattern matching is an NP-hard problem in the worst case. This paper presents a theoretical comparison of spatial pattern matching approaches, showing how the prominent methods compare for each type of spatial relation they support. We further highlight the common techniques used to gain performance improvements and provide suggestions towards developing approximate solutions to this form of spatial search.