In software engineering, benchmarks are widely used to evaluate and compare the performance, functionality, and reliability of analysis tools. Despite the prevalence of benchmarks in areas such as databases, machine learning, and programming languages, there is a notable absence of publicly available benchmarks for REST APIs, a cornerstone of modern web-based systems. While existing research papers occasionally employ similar REST APIs in their evaluations, opportunistic API selection hampers comparison. Moreover, these studies often rely on API documentation and structural characteristics. Without a reliable benchmark, API data used in evaluations may be outdated or inaccurate, compromising reliability and reproducibility. Hence, this paper addresses a gap in the literature by providing a comprehensive and Public REST API Benchmark (PRAB), to be utilized by researchers in their evaluations. The benchmark contains documentation and structural characteristics of 60 publicly available REST APIs. First, we conduct a systematic mapping study to discover the available and public REST APIs that are utilized in the academic literature. Then, by analyzing the resulting APIs, we report their structural characteristics (e.g., routes, query parameters, HTTP methods, authentication). Finally, we provide their documentation (i.e., OpenAPI Specification, Postman Collection) in a publicly available GitHub repository, to help with future evaluations of REST API studies.

APIs are increasingly prominent for modern web applications, allowing millions of users around the world to access data. Reducing the risk of API defects - and consequently failures - is key, notably for security, availability, and maintainability purposes. Documenting an API is crucial, allowing the user to better understand it. Moreover, API testing techniques often require formal documentation as input. However, documenting is a time-consuming and error-prone task, often overlooked by developers. Natural Language Processing (NLP) could assist API development, as recent Large Language Models (LLMs) demonstrated exceptional abilities to automate tasks based on their colossal training data. Data mining could also be utilized, synthesizing API information scattered across the web. Hence, I present my PhD project aimed at exploring the usage of NLP-related technologies and data mining to augment and validate APIs. The research questions of this PhD project are: (1) What types of APIs can benefit from NLP and data mining assistance? (2) What API problems can be solved with such methods? (3) How effective are the methods (i.e. LLMs) in assisting APIs? (4) How efficient are the methods in assisting APIs (i.e. time and costs)?