Software specifications are essential for many Software Engineering (SE) tasks such as bug detection and test generation. Many existing approaches are proposed to extract the specifications defined in natural language form (e.g., com-ments) into formal machine-readable form (e.g., first-order logic). However, existing approaches suffer from limited generalizability and require manual efforts. The recent emergence of Large Language Models (LLMs), which have been successfully applied to numerous SE tasks, offers a promising avenue for automating this process. In this paper, we conduct the first empirical study to evaluate the capabilities of LLMs for generating software specifications from software comments or documentation. We evaluate LLMs' performance with Few-Shot Learning (FSL) and compare the performance of 13 state-of-the-art LLMs with traditional approaches on three public datasets. In addition, we conduct a comparative diagnosis of the failure cases from both LLMs and traditional methods, identifying their unique strengths and weaknesses. Our study offers valuable insights for future research to improve specification generation.

Testing in various driving scenarios is one of the essential methods to enhance the reliability of autonomous driving systems (ADS). Existing ADS testing research has shown effectiveness in detecting safety violations by generating diverse driving scenarios. However, they do not consider the various geographical features and thus have limited ability to find safety violations caused by complex geographical features. Our paper addresses this limitation by analyzing a given high-definition map and collecting its geographical features. We leverage this information and develop a technique for generating corner case scenarios that exercise diverse geographical features such as curves and slopes. Our approach first generates the ego-vehicle’s driving routes so that they achieve full lane coverage on the entire map, then clusters those routes by geographical features, and constructs driving scenarios by adding other objects and environments. In our experiments on Autoware-Universe, we evaluate our technique with six high-definition maps from the Carla simulator. Our results show that driving scenarios generated by our tool effectively exercise more diverse geographical features than existing work. As a result, our tool uncovers new safety violations that are caused by complex geographical features and would not be detected by existing work.