The growing popularity and widespread use of software applications (apps) across various domains have driven rapid industry growth. Along with this growth, fast-paced market changes have led to constantly evolving software requirements. Such requirements are often grounded in feature requests and enhancement suggestions, typically provided by users in natural language (NL). However, these requests often suffer from defects such as ambiguity and incompleteness, making them challenging to interpret. Traditional validation methods (e.g., interviews and workshops) help clarify such defects but are impractical in decentralized environments like open-source software (OSS), where change requests originate from diverse users on platforms like GitHub. This paper proposes a novel approach leveraging Large Language Models (LLMs) to detect and refine NL defects in feature requests. Our approach automates the identification of ambiguous and incomplete requests and generates clarification questions (CQs) to enhance their usefulness for developers. To evaluate its effectiveness, we apply our method to real-world OSS feature requests and compare its performance against human annotations. In addition, we conduct interviews with GitHub developers to gain deeper insights into their perceptions of NL defects, the strategies they use to address these defects, and the impact of defects on downstream software engineering (SE) tasks.

The popularity and usage of software applications (apps) among users have encouraged the industry to grow throughout many domains and categories. In today's competitive market, the requirements for apps change rapidly and continuously. Further, these requirements can be introduced through constant feature requests by end-users & other stakeholders of apps through various sources. Such requests are expressed in Natural Language (NL), where they can introduce various types of ambiguity which can lead to misunderstandings between stakeholders, developers, and testers about what feature is actually requested. This can result in wasted time and effort as developers may implement features incorrectly or build the wrong functionality altogether. This proposal aims to tackle ambiguity in feature requests using large language models (LLMs). Ultimately, the objective is to utilize LLMs to develop a practical and interactive framework to support the requirements evaluation phase. Through this framework, the submitted feature & change requests are first evaluated to detect potential ambiguities. Then, interactive negotiations with users are initiated to address such deficiencies.

As mobile technology has advanced, individuals have started relying on their smartphones to conduct more of their everyday tasks. From playing games or streaming media to social networking and banking, apps on a user's device may have access to the most sensitive information on the device. Privacy policies are designed to inform users of such data practices so that they can make reasonable decisions when using the app. However, an app's true behavior may not always align with the statements in a privacy policy. In this work, we divide our study into two components and compare the viability of various large language models (LLMs): methods for extracting and summarizing privacy policy data practices, or information-type extraction and action-verb extraction; and methods for measuring whether the policy acknowledges the interaction with certain information (sensitive data) compared to identified methods within its app's source code. Fine-tuning GPT-3.5 Turbo delivers a higher average F1-score for both action verb extraction (0.50) and information-type extraction (0.84) compared to other LLMs. ChatGPT outperforms other language models in traditional semantic similarity, providing a consistently high performance, including the highest F1-score (0.52) for this task. Our approaches demonstrate that these LLMs are viable in performing such tasks and additionally that pre-trained instruction-based LLMs are capable of identifying the complex relationships between policies and source code.