User feedback on software usage is utilised by developers to improve their software. Software product forums are platforms rich in software-related user feedback, such as forum threads containing bug reports or requests for new features. However, previous studies have mainly focused on analysing user feedback from software product forums as individual sentences, which can lead to missing insights and a lack of understanding of the overall context of forum posts. To fill this gap in research, this work examines user feedback found in software product forum posts to investigate the differences between content classifications found in forum sentences and posts. We manually evaluated software product forum posts collected from two open-sourced software product forums and discovered five new types of user feedback that can only be identified when examining user feedback in the form of forum posts. Additionally, we examined the association between sentence classifications found within software product forums. Our results indicate that contextual information complimenting product improvement insights can be found in software product forums, with a confidence of 0.75 and 0.69 for the association between apparent bug and application usage sentences. This information can be used to reduce manual efforts required to chase up missing contextual information when attempting to understand or fix software issues. We also provide insights into the progression of posts in software product forums at the thread-level, and our progression flowchart can be used to summarise the sequence of events in software product forum threads. Our findings reveal the importance of looking at user feedback within software product forums in the format of forum posts to identify new insights on user feedback for software improvements. Editor’s note: Open Science material was validated by the Journal of Systems and Software Open Science Board.

User feedback is an important resource in modern software development, often containing requirements that help address user concerns and desires for a software product. The feedback in online channels is a recent focus for software engineering researchers, with multiple studies proposing automatic analysis tools. In this work, we investigate the product forums of two large open source software projects. Through a quantitative analysis, we show that forum feedback is often manually linked to related issue tracker entries and product documentation. By linking feedback to their existing documentation, development teams enhance their understanding of known issues, and direct their users to known solutions. We discuss how the links between forum, issue tracker, and product documentation form a requirements ecosystem that has not been identified in the previous literature. We apply state-of-the-art deep-learning to automatically match forum posts with related issue tracker entries. Our approach identifies requirement matches with a mean average precision of 58.9% and hit ratio of 82.2%. Additionally, we apply deep-learning using an innovative clustering technique, achieving promising performance when matching forum posts to related product documentation. We discuss the possible applications of these automated techniques to support the flow of requirements between forum, issue tracker, and product documentation.

Feedback from software users is vital for engineering better software requirements. One tool for extracting requirements from online user feedback is clustering, where the most mentioned topics are found by grouping similar feedback together. For these topics to be understood, clusters have been summarized in previous work using characterizing phrases or sentences. This work evaluates which method of characterization (unigrams, bigrams, trigrams, or sentences) is most effective for understanding the semantic meaning of a whole cluster using feedback from multiple feedback sources. We evaluate multiple characterization methods to determine the ability of each method to create distinct, descriptive characterizations. We further evaluate the amount of requirements relevant characterizations created by each characterization method. We find that unigrams, bigrams, trigrams, and full sentences all perform similarly in distinguishing clusters from each other. However, we find that fewer and more expressive characterizations, such as full sentences, contain more requirements relevant information from a feedback cluster compared to more numerous but less expressive unigrams, meaning a sentence will better summarize the important requirement relevant information from a cluster. Our findings inform the future development of user feedback clustering tools, with different cluster characterization methods being quantitatively measured for the first time.

The software industry lacks gender diversity. Recent research has suggested that a toxic working culture is to blame. Studies have found that communications in software repositories directed towards women are more negative in general. In this study, we use a destructive criticism lens to examine gender differences in software code review feedback. Software code review is a practice where code is peer reviewed and negative feedback is often delivered. We explore differences in perceptions, frequency, and impact of destructive criticism across genders. We surveyed 93 software practitioners eliciting perceived reactions to hypothetical scenarios (or vignettes) where participants are asked to imagine receiving either constructive or destructive criticism. In addition, the survey collected general opinions on feedback obtained during software code review as well as the frequency that participants give and receive destructive criticism. We found that opinions on destructive criticism vary. Women perceive destructive criticism as less appropriate and are less motivated to continue working with the developer after receiving destructive criticism. Destructive criticism is fairly common with more than half of respondents having received nonspecific negative feedback and nearly a quarter having received inconsiderate negative feedback in the past year. Our results suggest that destructive criticism in code review could be a contributing factor to the lack of gender diversity observed in the software industry.