Peer Reviewed

Given natural language test case description for an Android application, existing testing approaches require developers to manually write scripts using tools such as Appium and Espresso to execute the corresponding test case. This process is labor-intensive and demands significant effort to maintain as UI interfaces evolve throughout development. In this work, we introduce LELANTE, a novel framework that utilizes large language models (LLMs) to automate test case execution without requiring pre-written scripts. LELANTE interprets natural language test case descriptions, iteratively generate action plans, and perform the actions directly on the Android screen using its GUI. LELANTE employs a screen refinement process to enhance LLM interpretability, constructs a structured prompt for LLMs, and implements an action generation mechanism based on chain-of-thought reasoning of LLMs. To further reduce computational cost and enhance scalability, LELANTE utilizes model distillation using a foundational LLM. In experiments across 390 test cases spanning 10 popular Android applications, LELANTE achieved a 73% test execution success rate. Our results demonstrate that LLMs can effectively bridge the gap between natural language test case description and automated execution, making mobile testing more scalable and adaptable.

We explored the challenges practitioners face in software testing and proposed automated solutions to address these obstacles. We began with a survey of local software companies and 26 practitioners, revealing that the primary challenge is not writing test scripts but aligning testing efforts with business requirements. Based on these insights, we constructed a usecase $\rightarrow$ (high-level) test-cases dataset to train/fine-tune models for generating high-level test cases. High-level test cases specify what aspects of the software’s functionality need to be tested, along with the expected outcomes. We evaluated large language models, such as GPT-4o, Gemini, LLaMA 3.1 8B, and Mistral 7B, where fine-tuning (the latter two) yields improved performance. A final (human evaluation) survey confirmed the effectiveness of these generated test cases. Our proactive approach strengthens requirement-testing alignment and facilitates early test case generation to streamline development.

Background: As improving code review (CR) effectiveness is a priority for many software development organizations, projects have deployed CR analytics platforms to identify potential improvement areas. The number of issues identified, which is a crucial metric to measure CR effectiveness, can be misleading if all issues are placed in the same bin. Therefore, a finer-grained classification of issues identified during CRs can provide actionable insights to improve CR effectiveness. Although a recent work by Fregnan et al. proposed automated models to classify CR-induced changes, we have noticed two potential improvement areas – i) classifying comments that do not induce changes and ii) using deep neural networks (DNN) in conjunction with code context to improve performances. Aims: This study aims to develop an automated CR comment classifier that leverages DNN models to achieve a more reliable performance than Fregnan et al. Method: Using a manually labeled dataset of 1,828 CR comments, we trained and evaluated supervised learning-based DNN models leveraging code context, comment text, and a set of code metrics to classify CR comments into one of the five high-level categories proposed by Turzo and Bosu. Results: Based on our 10-fold cross-validation-based evaluations of multiple combinations of tokenization approaches, we found a model using CodeBERT achieving the best accuracy of 59.3%. Our approach outperforms Fregnan et al.'s approach by achieving 18.7% higher accuracy. Conclusion: In addition to facilitating improved CR analytics, our proposed model can be useful for developers in prioritizing code review feedback and selecting reviewers.

We present a novel approach - CLAA - for API aspect detection in API reviews that utilizes transformer models trained with a supervised contrastive loss objective function. We evaluate CLAA using performance and impact analysis. For performance analysis, we utilized a benchmark dataset on developer discussions collected from Stack Overflow and compare the results to those obtained using state-of-the-art transformer models. Our experiments show that contrastive learning can significantly improve the performance of transformer models in detecting aspects such as Performance, Security, Usability, and Documentation. For impact analysis, we performed empirical and developer study. On a randomly selected and manually labeled 200 online reviews, CLAA achieved 92% accuracy while the SOTA baseline achieved 81.5%. According to our developer study involving 10 participants, the use of Stack Overflow + CLAA resulted in increased accuracy and confidence during API selection. Replication package: https://github.com/disa-lab/Contrastive-Learning-API-Aspect-ASE2023.

Background. Studies on developer productivity and well-being find that the perceptions of productivity in a software team can be a socio-technical problem. Intuitively, problems and challenges can be better handled by managing expectations in software teams. Aim. Our goal is to understand whether the expectations of software developers vary towards diverse stakeholders in software teams. Method. We surveyed 181 professional software developers to understand their expectations from five different stakeholders: (1) organizations, (2) managers, (3) peers, (4) new hires, and (5) government and educational institutions. The five stakeholders are determined by conducting semi-formal interviews of software developers. We ask open-ended survey questions and analyze the responses using open coding. Results. We observed 18 multi-faceted expectations types. While some expectations are more specific to a stakeholder, other expectations are cross-cutting. For example, developers expect work-benefits from their organizations, but expect the adoption of standard software engineering (SE) practices from their organizations, peers, and new hires. Conclusion. Out of the 18 categories, three categories are related to career growth. This observation supports previous research that happiness cannot be assured by simply offering more money or a promotion. Among the most number of responses, we find expectations from educational institutions to offer relevant teaching and from governments to improve job stability, which indicate the increasingly important roles of these organizations to help software developers. This observation can be especially true during the COVID-19 pandemic.

Low-code software development (LCSD) is an emerging paradigm that combines minimal source code with interactive graphical interfaces to promote rapid application development. LCSD aims to democratize application development to software practitioners with diverse backgrounds. Given that LCSD is relatively a new paradigm, it is vital to learn about the challenges developers face during their adoption of LCSD platforms. The online developer forum, Stack Overflow (SO), is popular among software developers to ask for solutions to their technical problems. We observe a growing body of posts in SO with discussions of LCSD platforms. In this paper, we present an empirical study of around 5K SO posts (questions + accepted answers) that contain discussions of nine popular LCSD platforms. We apply topic modeling on the posts to determine the types of topics discussed. We find 13 topics related to LCSD in SO. The 13 topics are grouped into four categories: Customization, Platform Adoption, Database Management, and Third-Party Integration. More than 40% of the questions are about customization, i.e., developers frequently face challenges with customizing user interfaces or services offered by LCSD platforms. The topic "Dynamic Event Handling" under the "Customization" category is the most popular (in terms of average view counts per question of the topic) as well as the most difficult. It means that developers frequently search for customization solutions such as how to attach dynamic events to a form in low-code UI, yet most (75.9%) of their questions remain without an accepted answer. We manually label 900 questions from the posts to determine the prevalence of the topics' challenges across LCSD phases. We find that most of the questions are related to the development phase, and low-code developers also face challenges with automated testing.

The learning and usage of an API is supported by official documentation. Like source code, API documentation is itself a software product. Several research results show that bad design in API documentation can make the reuse of API features difficult. Indeed, similar to code smells or code anti-patterns, poorly designed API documentation can also exhibit `smells'. Such documentation smells can be described as bad documentation styles that do not necessarily produce an incorrect documentation but nevertheless make the documentation difficult to properly understand and to use. Recent research on API documentation has focused on finding content inaccuracies in API documentation and to complement API documentation with external resources (e.g., crowd-shared code examples). We are aware of no research that focused on the automatic detection of API documentation smells. This paper makes two contributions. First, we produce a catalog of five API documentation smells by consulting literature on API documentation presentation problems. We create a benchmark dataset of 1,000 API documentation units by exhaustively and manually validating the presence of the five smells in Java official API reference and instruction documentation. Second, we conduct a survey of 21 professional software developers to validate the catalog. The developers agreed that they frequently encounter all five smells in API official documentation and 95.2% of them reported that the presence of the documentation smells negatively affects their productivity. The participants wished for tool support to automatically detect and fix the smells in API official documentation. We develop a suite of rule-based, deep and shallow machine learning classifiers to automatically detect the smells. The best performing classifier BERT, a deep learning model, achieves F1-scores of 0.75 - 0.97.

SQL Injection attack is one of the oldest yet effective attacks for web applications. Even in 2020, applications are vulnerable to SQL Injection attacks. The developers are sup-posed to take precautions such as parameterizing SQL queries, escaping special characters, etc. However, developers, especially inexperienced ones, often fail to comply with such guidelines. There are quite a few SQL Injection detection tools to expose any unattended SQL Injection vulnerability in source code. However, to the best of our knowledge, very few works have been done to suggest a fix of these vulnerabilities in the source code. We have developed a learning-based approach that prepares abstraction of SQL Injection vulnerable codes from training dataset and clusters them using hierarchical clustering. The test samples are matched with a cluster of similar samples and a fix suggestion is generated. We have developed a manually validated training and test dataset from real-world projects of Java and PHP to evaluate our language-agnostic approach. The results establish the superiority of our technique over comparable techniques. The code and dataset are released publicly to encourage reproduction.

Typosquatting is a form of internet cybersquatting generated from the mistakes (typos) made by internet users while typing a website address. It often leads the user to another unintended website. Sometimes it is exploited by cybersquatters to attract website traffic by redirecting common typos of popular websites to some other sites with malicious contents. A possible solution is defensive registration of similar domains and redirecting requests to the original site. This would be affordable for the owner of the original domain if a short list of such probable typo domain names can be predicted. Existing works on typosquatting mostly try to detect typo sites by analyzing logs. However, to the best of our knowledge, none of them can predict probable typo variations of a given URL at pre-registration phase. In this paper, we present TypoWriter, an RNN based error prediction tool to fill this gap. TypoWriter achieves a good performance in terms of successful predictions that match with the ground-truth. It is compared with five widely used typo generation tools and substantial improvement is observed.

Software developers around the globe have shown tremendous interests in blockchain with more than seven thousand active blockchain software (BCS) projects on Github. Yet, little research has focused on understanding the challenges encountered by the developers of those projects as well as its' users. Therefore, the objective of this study is to better understand the primary areas of challenges encountered by the BCS community. Using a Latent Dirichlet Allocation based topic modeling, we identify discussion topics from the two Blockchain related StackExchange sites. We manually investigated the posts belonging to each topic to understand challenges encountered by the developers. The results of our study revealed that while the ratios of posts on BCS development are increasing, the ratios of posts on mining cryptocurrencies are decreasing. Due to the scarcity of expert blockchain developers, posts on BCS development are more likely to go either answered or encounter more delays than posts on other topics. Based on our findings, we recommend project maintainers to spend efforts to improve documentations on BCS development as the community lacks supporting materials on that area the most.

New programming languages (e.g., Swift, Go, Rust, etc.) are being introduced to provide a better opportunity to developers by matching the requirements of new platforms and application contexts. In the beginning, a programming language is likely to have constraints of resources that encourage the developers to seek help from experienced peers active in Question-answering (QA) sites such as Stack Overflow (SO). In this study, we would like to analyze the discussions on three popular new languages that are introduced after the inception of SO (2008). The relevant posts in SO present an interesting representation of the growth/evolution of that language and also expose the demands of the relevant development community. The major findings of the study are: (i) the time when adequate resources are expected to be available vary from language to language, (ii) the unanswered question ratio increases regardless of the age of the language and (iii) a new language is benefited from its predecessor language. The study outcome is likely to help the owner/sponsor of these languages to design better features and documentation and software developers or students to prepare themselves to work on these languages in an informed way.

With the advancement in big data, NoSQL databases are enjoying ever-growing popularity. The increasing use of this technology in large applications also brings security concerns to the fore. Historically, SQL injection has been one of the major security threats over the years. Recent studies reveal that NoSQL databases also have become vulnerable to injections. However, NoSQL security is yet to receive the attention it deserves from the industry or academia. In this work, we develop a tool for detecting NoSQL injections using supervised learning. To the best of our knowledge, our developed training dataset on NoSQL injection is the first of its kind. We manually design important features and apply various supervised learning algorithms. Our tool has achieved 0.93 F2-score as established by 10-fold cross-validation. We also apply our tool to a NoSQL injection generating tool, NoSQLMap and find that our tool outperforms Sqreen, the only available NoSQL injection detection tool, by 36.25% in terms of detection rate. The proposed technique is also shown to be database-agnostic achieving similar performance with injection on MongoDB and CouchDB databases.

StackOverflow (SO), the most popular community Q&A site rewards answerers with reputation scores to encourage answers from volunteer participants. However, irrespective of the difficulty of a question, the contributor of an accepted answer is awarded with the same 'reputation' score, which may demotivate an user's additional efforts to answer a difficult question. To facilitate a question difficulty aware rewarding system, this study proposes SOQDE (Stack Overflow Question Difficulty Estimation), a supervised learning based Question difficulty estimation model for the StackOverflow. To design SOQDE, we randomly selected 936 questions from a SO datadump exported during September 2017. Two of the authors independently labeled those questions into three categories (basic, intermediate, or advanced), where conflicting labels were resolved through tie-breaking votes from a third author. We performed an empirical study to determine how the difficulty of a question impacts its outcomes, such as number of votes, resolution time, and number of votes. Our results suggest that the answers of a basic question receive more votes and therefore would generate more reputation points for an answerer. Due to less incentives relative to efforts spent by an answerer, intermediate and advanced questions encounter significantly more delays than the basic questions, which further validates the need of a model like SOQDE. To build our model, we have identified textual and contextual features of a question and divided them into two categories-pre-hoc and post-hoc features. We observed a model based on Random Forest achieving the highest mean accuracy (67.6%), using only answer-independent pre-hoc features. Accommodating answer-dependent post-hoc features, we were able to improve the mean accuracy of our model to 75.2%.

Background: The application of the blockchain technology has shown promises in various areas, such as smart-contracts, Internet of Things, land registry management, identity management, etc. Although Github currently hosts more than three thousand active blockchain software (BCS) projects, a few software engineering research has been conducted on their software engineering practices. Aims: To bridge this gap, we aim to carry out the first formal survey to explore the software engineering practices including requirement analysis, task assignment, testing, and verification of blockchain software projects. Method: We sent an online survey to 1,604 active BCS developers identified via mining the Github repositories of 145 popular BCS projects. The survey received 156 responses that met our criteria for analysis. Results: We found that code review and unit testing are the two most effective software development practices among BCS developers. The results suggest that the requirements of BCS projects are mostly identified and selected by community discussion and project owners which is different from requirement collection of general OSS projects. The results also reveal that the development tasks in BCS projects are primarily assigned on voluntary basis, which is the usual task assignment practice for OSS projects. Conclusions: Our findings indicate that standard software engineering methods including testing and security best practices need to be adapted with more seriousness to address unique characteristics of blockchain and mitigate potential threats.

Sentiment Analysis tools, developed for analyzing social media text or product reviews, work poorly on a Software Engineering (SE) dataset. Since prior studies have found developers expressing sentiments during various SE activities, there is a need for a customized sentiment analysis tool for the SE domain. On this goal, we manually labeled 2000 review comments to build a training dataset and used our dataset to evaluate seven popular sentiment analysis tools. The poor performances of the existing sentiment analysis tools motivated us to build SentiCR, a sentiment analysis tool especially designed for code review comments. We evaluated SentiCR using one hundred 10-fold cross-validations of eight supervised learning algorithms. We found a model, trained using the Gradient Boosting Tree (GBT) algorithm, providing the highest mean accuracy (83%), the highest mean precision (67.8%), and the highest mean recall (58.4%) in identifying negative review comments.

Participatory sensing is a revolutionary new paradigm where citizens voluntarily sense their surroundings using readily available sensing devices such as mobile phones and share this information for mutual benefit of community members. To encourage ample participation of users, ensuring their privacy is inevitable. Existing techniques that attempt to protect location privacy with spatial cloaking suffer from irrecoverable data quality degradation. To the best of our knowledge, very few works provided a solution preserving high data quality/utility at the destination server, however, suffered from unacceptable computational overhead. This paper presents an improved deterministic alternative and also a faster variant by exploiting several optimization issues. Theoretical formulations and extensive simulation results are presented to establish the applicability of our proposed techniques.

In passive localization, sensors try to locate an event without any knowledge of event's emitted power. So, this is a more challenging problem compared to active localization. Existing passive localization schemes use expensive and noise-vulnerable range-based techniques. In this paper, we propose, to the best of our knowledge for the first time, a cost-effective range-free passive localization scheme exploiting hybrid sensor network model where mobile sensors are deployed on demand once an event is sensed by a static sensor. Efficient use of mobile sensors leads to two concomitant optimization problems: (1) positioning the mobile sensors so that the expected possible event location area is minimized; and (2) minimizing their overall traversed distance. To solve the first problem, we have developed a novel arc-coding based range-free localization technique that can accurately define the area of possible event location from the feedback of arbitrarily placed sensors without relying on expensive hardware to estimate range of signals. We have achieved significantly high localization accuracy with a low number of mobile sensors even after considering significant environmental noise. To solve the second problem, three alternative deployment strategies for the mobile sensors were simulated to recommend the best.

Success of participatory sensing system depends on the extent of voluntary participation by users. To increase participation, incentive such as rewards can be used only if reported data has associated user identification. This creates serious threat to participating users' location privacy. Existing techniques tried to solve it with spatial clocking, which suffers from inferior data integrity. In this paper, we present a subset coding based anonymization scheme that can safeguard users' location privacy with k-anonymity while preserving almost lossless data integrity at the destination server. Adversary threats to our scheme are comprehensively analyzed to develop robust strategies and analytical bounds on system parameters for location privacy risk mitigation. Applicability of the proposed scheme is established with extensive simulation results.

In participatory sensing system community people contribute information to be shared by everybody. However, none would be tolerant enough to contribute voluntarily if her privacy is not protected. This has evoked the idea of research in the area of preserving privacy in participatory sensing system. On the other hand, data integrity is desired imperatively to make the service trustworthy and user-friendly. In this paper, we have investigated the performance of a greedy algorithm and its randomized variant to achieve an acceptable tradeoff between these two orthogonal key parameters. We have also analyzed the ability of a third party adversary to decode privacy-sensitive data by eavesdropping. Our experimental results show that the proposed method is performing satisfactorily as an approach of balancing user privacy and data integrity.

In a decentralized sensor network, localization process relies on the integrity of participating sensors. Existence of malicious beacon nodes in the vicinity of non-beacon nodes affects this process. This paper presents a trilateration-based secure localization technique, which is capable of estimating the location of a sensor with high accuracy so long four neighbouring beacon nodes are benign, irrespective of the number of neighbouring liars and without assuming any trust model. In realistic scenarios of wireless environment where transmitted signals attenuate randomly due to fading, the liar-tolerance level of this attack-resistant technique has to be relaxed accordingly. Superiority of this technique against the state-of-the-art has been established with extensive simulation results in terms of location estimation accuracy and liar-filtering probability.