Abstract Expedient design choices in software development can lead to Technical Debt (TD), with development teams documenting such decisions as Self-Admitted TD (SATD). Algorithm Debt (AD) is a type of TD resulting from the suboptimal implementation of algorithms, which impacts system performance. Given the impact of AD, its automated detection is crucial in Deep Learning (DL) frameworks due to their complexity and evolution. Early detection of AD in DL frameworks can help mitigate model degradation and scalability issues. Despite previous studies on the automated detection of TD from SATD using Machine Learning (ML)/DL models, research on AD detection in DL frameworks remains underexplored. In this study, we empirically investigated the performance of ML/DL models for the automated detection of AD using a dataset of 38, 881 SATD comments from seven DL frameworks. We trained, evaluated, and tested ML/DL models, used embeddings from both DL and large language models, and explored an approach to enrich the dataset with handcrafted features based on AD-related keywords. Our findings reveal that AD is frequently misclassified as Design or Implementation Debt. Logistic Regression (an ML model) with Custom AD Features, achieved an F1-score of 54% for AD, outperforming other ML/DL models (42% to 52%), highlighting the importance of tailored feature engineering. Our research advances automated AD detection in DL frameworks by providing insights into the strengths and limitations of ML/DL models, serving as a first step to guide future tool development. This could help developers using DL frameworks to identify AD issues during development, thereby enhancing system reliability by mitigating model degradation and scalability challenges.

Ruby is a widely used open-source programming language, valued for its simplicity, especially in web development. Despite its popularity, with over one million users on GitHub, little is known about the issues faced by Ruby developers. This study aims to investigate the key topics, trends, and difficulties faced by Ruby developers by analyzing over 498,000 Ruby-related questions on Stack Overflow (SO), followed by a survey of 154 Ruby developers. We employed BERTopic modeling and manual analysis to develop a taxonomy of 35 topics, grouped into six main categories. Our findings reveal that Web Application Development is the most commonly discussed category, while Ruby Gem Installation and Configuration Issues emerged as the most challenging topic. Analysis of trends on SO showed a steady decline. A survey of 154 Ruby developers demonstrated that \(31.6\%\) of the participants find the Core Ruby Concepts category particularly difficult, while Application Quality and Security is found to be difficult for over \(40\%\) of experienced developers. Notably, a comparison between survey responses and SO metrics highlights a misalignment, suggesting that perceived difficulty and objective indicators from SO differ; emphasizing the need for improved metrics to capture developer challenges better. Our study provides insights about the challenges Ruby developers face and strong implications for researchers.

Mentorship in Open Source Software (OSS) projects is crucial for reducing barriers to entry for newcomers and for fostering the technical and social integration of new contributors. While mentorship in OSS has been recognized as essential for sustainable project growth, quantitative research supporting qualitative findings is not common. To address this gap, we present OpenMent, a comprehensive dataset comprising over 500,000 issue comments, pull request comments, and commit messages from GitHub projects participating in the Google Summer of Code (GSoC) program. OpenMent is curated to capture role-specific interactions and communication patterns between mentors and mentees, providing information on the challenges and dynamics of OSS mentoring. This dataset is designed to be a reusable resource for the Software Engineering community, enabling researchers and practitioners to explore mentorship dynamics and investigate the impact of mentoring on contributor retention. By making OpenMent openly available, we aim to facilitate future research in OSS mentorship, fostering a deeper understanding of mentorship challenges, strategies, and contributions to the growth and inclusivity of OSS ecosystems.

Modern Code Review (MCR) is an essential practice in software engineering, supporting early defect detection, enhancing code quality, and fostering knowledge. To manage code review tasks effectively, developers need to understand the intent behind code changes, such as a bug fix, test, refactoring, or new feature. Traditional methods for categorizing code changes in MCR rely on rule-based heuristics with predefined keywords. However, these methods lack context regarding the code changes, leading to limited generalizability, particularly when dealing with sparsely documented changes. This paper addresses these limitations by investigating the potential of Large Language Models (LLMs) for changes' intent classification. We introduce LLM Change Classifier (LLMCC), an LLM-based approach that classifies code changes based on their underlying intent. We evaluate the effectiveness of LLMCC by conducting an empirical study on three open-source projects: Android, OpenS tack, and Qt. The performance of LLMCC was benchmarked against traditional heuristic methods, conventional machine learning algorithms (including Decision Trees and Random Forests), and state-of-the-art transformer models (including BERT and RoBERTa). Results show that LLMCC significantly enhances code change intent classification accuracy, achieving up to a 33 % improvement in F1 score over heuristic-based methods. Additionally, LLMCC outperformed both traditional machine learning and transformer models, achieving an average 77% improvement in terms of Matthew Correlation Coefficient (MCC). These findings underscore the potential of LLMCC to streamline code change intent classification.

Programming languages can benefit from one another by utilizing a pre-trained model for software engineering tasks such as code summarization and method name prediction. While full fine-tuning of Code Language Models (Code-LMs) has been explored for multilingual knowledge transfer, research on Parameter Efficient Fine-Tuning (PEFT) for this purpose is lim-ited. AdapterFusion, a PEFT architecture, aims to enhance task performance by leveraging information from multiple languages but primarily focuses on the target language. To address this, we propose AdvFusion, a novel PEFT-based approach that effectively learns from other languages before adapting to the target task. Evaluated on code summarization and method name prediction, AdvFusion outperforms AdapterFusion by up to 1.7 points and surpasses LoRA with gains of 1.99, 1.26, and 2.16 for Ruby, JavaScript, and Go, respectively. We open-source our scripts for replication purposes11https://github.com/ist1373/AdvFusion.

Large language models (LLMs) have significantly improved their ability to perform tasks in the field of code generation. However, there is still a gap between LLMs being capable coders and being top-tier software engineers. The most recent trend is using LLM-based agents to iterate the code generation process. Based on the observation that top-level software engineers often ask clarifying questions to reduce Ambiguity in both requirements and coding solutions, we argue that the same should be applied to LLMs for code generation tasks. For this purpose, we define the communication skills of LLMs as “being able to ask clarifying questions when the description of the code generation problem has issues”. In this study, we restrict these issues to three matters from the software requirement engineering field: inconsistent requirements, ambiguous requirements, and incomplete requirements. By asking probing questions about the requirements of problem descriptions before generating the final code, the challenges of programming with LLMs, such as unclear intent specification may be alleviated, resulting to a correct code in the initial iterations. In this work, we conducted an empirical study on the benchmark and analysis of the communication skills of LLMs for code generation. We created a new benchmark, HumanEvalComm, by modifying problem descriptions according to three issues mentioned above, Inconsistency , Ambiguity , Incompleteness . We then experimented on HumanEvalComm with different Code LLMs, and a new LLM agent approach, C o de C l a rificatio n a nd G eneration A ge n t (Okanagan), to identify and ask questions in ambiguous parts from code and descriptions for further refining the generated code. In the evaluation, we introduced an LLM-based evaluator and created Communication Rate and Good Question Rate as the evaluation metrics to represent the ratio of questions asked and questions with good quality in responses. We found that more than 60% of responses from Code LLMs still generate code rather than ask questions when the problem descriptions are manually modified according to different clarification categories. The Pass@1 and Test Pass Rate of most Code LLMs drop by 35% \(\sim\) 52% and by 17% \(\sim\) 35% respectively, with statistical significance in each category for over 75% numbers. Okanagan, as an LLM agent approach that uses LLM such as ChatGPT 3.5, effectively increases the Communication Rate and Good Question Rate by an absolute 58% and 38%, respectively. Thus, Okanagan boosts Pass@1 and Test Pass Rate by an absolute 8% and 7%, respectively, when the problem descriptions are modified based on given clarification categories. This result indicates the potential for achieving more effective communication capability using LLM agent. Our benchmark and full code are publicly available at https://github.com/jie-jw-wu/human-eval-comm .

Recent advancements in developing Pre-trained Language Models for Code (Code-PLMs) have urged many areas of Software Engineering (SE) and brought breakthrough results for many SE tasks. Though these models have achieved the state-of-the-art performance for SE tasks for many popular programming languages, such as Java and Python, the Scientific Software and its related languages like R programming language have rarely benefited or even been evaluated with the Code-PLMs. Research has shown that R has many differences with other programming languages and requires specific techniques. In this study, we provide the first insights for code intelligence for R. For this purpose, we collect and open source an R dataset, and evaluate Code-PLMs for the two tasks of code summarization and method name prediction using several settings and strategies, including the differences in two R styles, Tidy-verse and Base R. Our results demonstrate that the studied models have experienced varying degrees of performance degradation when processing R programming language code, which is supported by human evaluation. Additionally, not all models show performance improvement in R-specific tasks even after multi-language fine-tuning. The dual syntax paradigms in R significantly impact the models’ performance, particularly in code summarization tasks. Furthermore, the project-specific context inherent in R codebases significantly impacts the performance when attempting cross-project training. Interestingly, even when Large Language Models like CodeLlama and StarCoder2 are used for code generation, the Pass@K ( \(K = 1,5,10\) ) results lag significantly behind Python scores. Our research shows that R as a low-resource language requires different techniques to collect a high-quality data. Specifically separating the two R styles has a great impact on the results and the separate dataset could increase the performance of the models. Our research sheds light on the capabilities of Code-PLMs and opens new research directions for researchers and practitioners for developing code intelligence tools and techniques for R. With R’s widespread use and popularity, the results of our study can potentially benefit a large community of R developers, both in research and industry.

In this report, we introduce the dataset curated to replicate and extend experiments on R programming tasks, particularly code summarization and method name prediction. The dataset was generated by collecting R repositories from GitHub, parsing the code snippets using the tree-sitter parser, and matching them with natural language descriptions based on Roxygen2 documentation. Building on this dataset, our work conducts an in-depth analysis of the performance of Pre-trained Language Models for Code (Code-PLMs) on R code. We highlight the challenges posed by R’s dual paradigms—Tidyverse and Base R—and demonstrate that current models, including Large Language Models, exhibit varying degrees of performance degradation when applied to R code. As a result, we underscore the complexity of effectively leveraging Code-PLMs for R, given its diverse programming styles and language features.

Developers spend much time finding information that is relevant to their questions. Stack Overflow has been the leading resource, and with the advent of Large Language Models (LLMs), generative models such as ChatGPT are used frequently. However, there is a catch in using each one separately. Searching for answers is time-consuming and tedious, as shown by the many tools developed by researchers to address this issue. On the other, using LLMs is not reliable, as they might produce irrelevant or unreliable answers (i.e., hallucination). In this work, we present StackRAG, a retrieval-augmented Multiagent generation tool based on LLMs that combines the two worlds: aggregating the knowledge from SO to enhance the reliability of the generated answers. Initial evaluations show that the generated answers are correct, accurate, relevant, and useful. A description video can be found here11Please note that as the tool requires API keys, we are not able to set up a live demo of StackRAG..

Large Language Models (LLMs) have demonstrated remarkable success in various natural language processing and software engineering tasks, such as code generation. The LLMs are mainly utilized in the prompt-based zero/few-shot paradigm to guide the model in accomplishing the task. GPT-based models are one of the popular ones studied for tasks such as code comment generation or test generation. These tasks are 'generative' tasks. However, there is limited research on the usage of LLMs for 'non-generative' tasks such as classification using the prompt-based paradigm. In this preliminary exploratory study, we investigated the applicability of LLMs for Code Clone Detection (CCD), a non-generative task. By building a mono-lingual and cross-lingual CCD dataset derived from CodeNet, we first investigated two different prompts using ChatGPT to detect Type-4 code clones in Java-Java and Java-Ruby pairs in a zero-shot setting. We then conducted an analysis to understand the strengths and weaknesses of ChatGPT in CCD. ChatGPT surpasses the baselines in cross-language CCD attaining an F1-score of 0.877 and achieves comparable performance to fully fine-tuned models for mono-lingual CCD, with an F1-score of 0.878. Also, the prompt and the difficulty level of the problems has an impact on the performance of ChatGPT. Finally, we provide insights and future directions based on our initial analysis1.

Pre-trained Code Language Models (Code-PLMs) have shown many advancements and achieved state-of-the-art results for many software engineering tasks in the past few years. These models are mainly targeted at popular programming languages such as Java and Python, leaving out many others like R. Though R has a wide community of developers and users, there is little known about the applicability of Code-PLMs for R. In this preliminary study, we aim to investigate the vulnerability of Code-PLMs for code entities in R. For this purpose, we use an R dataset of code and comment pairs and then apply CodeAttack, a black-box attack model that uses the structure of code to generate adversarial code samples. We investigate how the model can attack different entities in R. This is the first step towards understanding the importance of R token types, compared to popular programming languages (e.g., Java). We limit our study to code summarization. Our results show that the most vulnerable code entity is the identifier, followed by some syntax tokens specific to R. The results can shed light on the importance of token types and help in developing models for code summarization and method name prediction for the R language.

In Stack Overflow (SO), a post consists of multiple components: title, question, answers, question tags, and comments. Developers can create any of these components and make changes, which we call 'edits'. Edits are an important aspect of QA websites to ensure the quality and correctness of the texts. We performed multiple analyses on the revision history of 23 million SO posts from 2008 to 2023, and we gain a more comprehensive understanding of developers' content maintenance behaviors which lay the foundation for further research.

Code Clone Detection (CCD) is a software engineering task that is used for plagiarism detection, code search, and code comprehension. Recently, deep learning-based models have achieved an F1-Score (a metric used to assess classifiers) of $$\sim $$ 95% on the CodeXGLUE benchmark. These models require many training data, mainly fine-tuned on Java or C++ datasets. However, no previous study evaluates the generalizability of these models where a limited amount of annotated data is available. The main objective of this research is to assess the ability of the CCD models as well as few-shot learning algorithms for unseen programming problems and new languages (i.e., the model is not trained on these problems/languages). We assess the generalizability of the state-of-the-art models for CCD in few-shot settings (i.e., only a few samples are available for fine-tuning) by setting three scenarios: i) unseen problems, ii) unseen languages, iii) combination of new languages and new problems. We choose CodeNet and conduct our experiments on Java, C++, and Ruby languages. Then, we employ Model Agnostic Meta-learning (MAML), where the model learns a meta-learner capable of extracting transferable knowledge from the train set; so that the model can be fine-tuned using a few samples. Finally, we combine contrastive learning with MAML to further study whether it can improve the results of MAML. Our results show that the performance of the models drops $$\sim 50\%$$ for Java and $$\sim 20\%$$ for C++ and Ruby for unseen problems, which are then boosted by $$13\%$$ to $$24\%$$ F1 scores for Java and C++/Ruby, respectively when MAML is used. Similar observations are found for unseen languages and the third scenario. Though in case of third scenario (i.e., unseen problems and unseen languages) the scores are lower. Integrating contrastive learning with MAML did not help in boosting the performance more than what we could achieve with MAML. Our results open new avenues of research and the need to develop robust models for clone detection, in the settings we investigated here.

Technical Debt (TD) is the implied cost of additional rework caused by choosing easier solutions in favour of shorter release time. It impacts software maintainability and evolvability, manifesting as different types (e.g., Code, Test, Architecture). Algorithm Debt (AD) is a new TD type recently identified as sub-optimal implementations of algorithm logic in scientific and Artificial Intelligence (AI) software. Given its newness, AD and its impact on AI-driven software remains a research gap. This poster aims to motivate reflective discussion on AD in AI software, by summarising findings, discussing its possible impact, and outlining future areas of work.

Code comments are significantly helpful in comprehending software programs and also aid developers to save a great deal of time in software maintenance. Code comment generation aims to automatically predict comments in natural language given a code snippet. Several works investigate the effect of integrating external knowledge on the quality of generated comments. In this study, we propose a solution, namely APIContext2Com, to improve the effectiveness of generated comments by incorporating the pre-defined Application Programming Interface (API) context. The API context includes the definition and description of the pre-defined APIs that are used within the code snippets. As the detailed API information expresses the functionality of a code snippet, it can be helpful in better generating the code summary. We introduce a seq-2-seq encoder-decoder neural network model with different sets of multiple encoders to effectively transform distinct inputs into target comments. A ranking mechanism is also developed to exclude non-informative APIs, so that we can filter out unrelated APIs. We evaluate our approach using the Java dataset from CodeSearchNet. The findings reveal that the proposed model improves the best baseline by 1.88 (8.24%), 2.16 (17.58% 1.38 (18.3%), 0.73 (14.17%), 1.58 (14.98 %) and 1.9 (6.92 %) for BLEU1, BLEU2, BLEU3, BLEU4, METEOR, ROUGE-L respectively. Human evaluation and ablation studies confirm the quality of the generated comments and the effect of architecture and ranking APIs.

Pre-trained Programming Language Models (PPLMs) achieved many recent states of the art results for many code-related software engineering tasks. Though some studies use data flow or propose tree-based models that utilize Abstract Syntax Tree (AST), most PPLMs do not fully utilize the rich syntactical information in source code. Still, the input is considered a sequence of tokens. There are two issues; the first is computational inefficiency due to the quadratic relationship between input length and attention complexity. Second, any syntactical information, when needed as an extra input to the current PPLMs, requires the model to be pre-trained from scratch, wasting all the computational resources already used for pre-training the current models. In this work, we propose Named Entity Recognition (NER) adapters, lightweight modules that can be inserted into Transformer blocks to learn type information extracted from the AST. These adapters can be used with current PPLMs such as CodeBERT, GraphCodeBERT, and CodeT5. We train the NER adapters using a novel Token Type Classification objective function (TTC). We insert our proposed work in CodeBERT, building CodeBERTER, and evaluate the performance on two tasks of code refinement and code summarization. CodeBERTER improves the accuracy of code refinement from 16.4 to 17.8 while using 20% of training parameter budget compared to the fully fine-tuning approach, and the BLEU score of code summarization from 14.75 to 15.90 while reducing 77% of training parameters compared to the fully fine-tuning approach.

A recent study by Ahmed and Devanbu reported that using a corpus of code written in multilingual datasets to fine-tune multilingual Pre-trained Language Models (PLMs) achieves higher performance as opposed to using a corpus of code written in just one programming language. However, no analysis was made with respect to fine-tuning monolingual PLMs. Furthermore, some programming languages are inherently different and code written in one language usually cannot be interchanged with the others, i.e., Ruby and Java code possess very different structure. To better understand how monolingual and multilingual PLMs affect different programming languages, we investigate 1) the performance of PLMs on Ruby for two popular Software Engineering tasks: Code Summarization and Code Search, 2) the strategy (to select programming languages) that works well on fine-tuning multilingual PLMs for Ruby, and 3) the performance of the fine-tuned PLMs on Ruby given different code lengths.

Pre-trained neural Language Models (PTLM), such as CodeBERT, are recently used in software engineering as models pre-trained on large source code corpora. Their knowledge is transferred to downstream tasks (e.g. code clone detection) via fine-tuning. In natural language processing (NLP), other alternatives for transferring the knowledge of PTLMs are explored through using adapters, compact, parameter efficient modules inserted in the layers of the PTLM. Although adapters are known to facilitate adapting to many downstream tasks compared to fine-tuning the model that require retraining all of the models' parameters- which owes to the adapters' plug and play nature and being parameter efficient-their usage in software engineering is not explored.

Developers frequently use APIs to implement certain functionalities, such as parsing Excel Files, reading and writing text files line by line, etc. Developers can greatly benefit from automatic API usage sequence generation based on natural language queries for building applications in a faster and cleaner manner. Existing approaches utilize information retrieval models to search for matching API sequences given a query or use RNN-based encoder-decoder to generate API sequences. As it stands, the first approach treats queries and API names as bags of words. It lacks deep comprehension of the semantics of the queries. The latter approach adapts a neural language model to encode a user query into a fixed-length context vector and generate API sequences from the context vector.

It is not a trivial problem to collect API-relevant examples, usages, and mentions on venues such as Stack Overflow. It requires efforts to correctly recognize whether the discussion refers to the API method that developers/tools are searching for. The content of the Stack Overflow thread, which consists of both text paragraphs describing the involvement of the API method in the discussion and the code snippets containing the API invocation, may refer to the given API method. Leveraging this observation, we develop ARSeek, a context-specific algorithm to capture the semantic and syntactic information of the paragraphs and code snippets in a discussion. ARSeek combines a syntactic word-based score with a score from a predictive model fine-tuned from CodeBERT. In terms of F1-score, ARSeek achieves an average score of 0.8709 and beats the state-of-the-art approach by 14%.

Many recent models in software engineering introduced deep neural models based on the Transformer architecture or use transformer-based Pre-trained Language Models (PLM) trained on code. Although these models achieve the state of the arts results in many downstream tasks such as code summarization and bug detection, they are based on Transformer and PLM, which are mainly studied in the Natural Language Processing (NLP) field. The current studies rely on the reasoning and practices from NLP for these models in code, despite the differences between natural languages and programming languages. There is also limited literature on explaining how code is modeled.

Code comment generation is the task of generating a high-level natural language description for a given code method/function. Although researchers have been studying multiple ways to generate code comments automatically, previous work mainly considers representing a code token in its entirety semantics form only (e.g., a language model is used to learn the semantics of a code token), and additional code properties such as the tree structure of a code are included as an auxiliary input to the model. There are two limitations: 1) Learning the code token in its entirety form may not be able to capture information succinctly in source code, and 2) The code token does not contain additional syntactic information, inherently important in programming languages.

Abstract Self-Admitted Technical Debt (SATD) is primarily studied in Object-Oriented (OO) languages and traditionally commercial software. However, scientific software coded in dynamically-typed languages such as R differs in paradigm, and the source code comments’ semantics are different (i.e., more aligned with algorithms and statistics when compared to traditional software). Additionally, many Software Engineering topics are understudied in scientific software development, with SATD detection remaining a challenge for this domain. This gap adds complexity since prior works determined SATD in scientific software does not adjust to many of the keywords identified for OO SATD, possibly hindering its automated detection. Therefore, we investigated how classification models (traditional machine learning, deep neural networks, and deep neural Pre-Trained Language Models (PTMs)) automatically detect SATD in R packages. This study aims to study the capabilities of these models to classify different TD types in this domain and manually analyze the causes of each in a representative sample. Our results show that PTMs (i.e., RoBERTa) outperform other models and work well when the number of comments labelled as a particular SATD type has low occurrences. We also found that some SATD types are more challenging to detect. We manually identified sixteen causes, including eight new causes detected by our study. The most common cause was failure to remember, in agreement with previous studies. These findings will help the R package authors automatically identify SATD in their source code and improve their code quality. In the future, checklists for R developers can also be developed by scientific communities such as rOpenSci to guarantee a higher quality of packages before submission.

Abstract Context Tangled commits are changes to software that address multiple concerns at once. For researchers interested in bugs, tangled commits mean that they actually study not only bugs, but also other concerns irrelevant for the study of bugs. Objective We want to improve our understanding of the prevalence of tangling and the types of changes that are tangled within bug fixing commits. Methods We use a crowd sourcing approach for manual labeling to validate which changes contribute to bug fixes for each line in bug fixing commits. Each line is labeled by four participants. If at least three participants agree on the same label, we have consensus. Results We estimate that between 17% and 32% of all changes in bug fixing commits modify the source code to fix the underlying problem. However, when we only consider changes to the production code files this ratio increases to 66% to 87%. We find that about 11% of lines are hard to label leading to active disagreements between participants. Due to confirmed tangling and the uncertainty in our data, we estimate that 3% to 47% of data is noisy without manual untangling, depending on the use case. Conclusion Tangled commits have a high prevalence in bug fixes and can lead to a large amount of noise in the data. Prior research indicates that this noise may alter results. As researchers, we should be skeptics and assume that unvalidated data is likely very noisy, until proven otherwise.

Automatic rumor detection is the task of finding rumors on social networks. Previous techniques leveraged the propagation structure of tweets to detect the rumors, which makes the propagation of tweets necessary to detect rumors. However, current text-based works provide sub-optimal results as compared to propagation-based techniques. This work presents a retrieval-based framework that leverages the similar tweets from the given train set and chooses the best model from an ensemble of models to predict the test tweet label. Our proposed framework is based on transformers-based pre-trained models (PTM's). Experiments on two public data sets used in previous works, show that our framework can detect the tweets with equivalent accuracy as propagation-based techniques. The primary advantage of this work is in early rumor detection. The proposed framework can detect rumors in few minutes compared to propagation-based works, which requires a significant amount of propagation of tweets that can take hours before they can be detected.

Code comments can help in program comprehension and are considered as important artifacts to help developers in software maintenance. However, the comments are mostly missing or are outdated, specially in complex software projects. As a result, several automatic comment generation models are developed as a solution. The recent models explore the integration of external knowledge resources such as Unified Modeling Language class diagrams to improve the generated comments. In this paper, we propose API2Com, a model that leverages the Application Programming Interface Documentations (API Docs) as a knowledge resource for comment generation. The API Docs include the description of the methods in more details and therefore, can provide better context in the generated comments. The API Docs are used along with the code snippets and Abstract Syntax Trees in our model.We apply the model on a large Java dataset of over 130,000 methods and evaluate it using both Transformer and RNN- base architectures. Interestingly, when API Docs are used, the performance increase is negligible. We therefore run different experiments to reason about the results. For methods that only contain one API, adding API Docs improves the results by 4% BLEU score on average (BLEU score is an automatic evaluation metric used in machine translation). However, as the number of APIs that are used in a method increases, the performance of the model in generating comments decreases due to long documentations used in the input. Our results confirm that the API Docs can be useful in generating better comments, but, new techniques are required to identify the most informative ones in a method rather than using all documentations simultaneously.

Context: Technical Debt (TD) is a metaphor used to describe code that is "not quite right." Although TD studies have gained momentum, TD has yet to be studied as thoroughly in non-Object-Oriented (OO) or scientific software such as R. R is a multi-paradigm programming language, whose popularity in data science and statistical applications has amplified in recent years. Due to R's inherent ability to expand through user-contributed packages, several community-led organizations were created to organize and peer-review packages in a concerted effort to increase their quality. Nonetheless, it is well-known that most R users do not have a technical programming background, being from multiple disciplines. Objective: The goal of this study is to investigate TD in the documentation of the peer-review of R packages led by rOpenSci. Method: We collected over 5,000 comments from 157 packages that had been reviewed and approved to be published at rOpenSci. We manually analyzed a sample dataset of these comments posted by package authors, editors of rOpenSci, and reviewers during the review process to investigate the types of TD present in these reviews. Results: The findings of our study include (i) a taxonomy of TD derived from our analysis of the peer-reviews (ii) documentation debt as being the most prevalent type of debt (iii) different user roles are concerned with different types of TD. For instance, reviewers tend to report some types of TD more than other roles, and the types of TD they report are different from those reported by the authors of a package. Conclusion: TD analysis in scientific software or peer-review is almost non-existent. Our study is a pioneer but within the context of R packages. However, our findings can serve as a starting point for replication studies, given our public datasets, to perform similar analyses in other scientific software or to investigate the rationale behind our findings.

Analysis of mobile app reviews has shown its important role in requirement engineering, software maintenance and evolution of mobile apps. Mobile app developers check their users' reviews frequently to clarify the issues experienced by users or capture the new issues that are introduced due to a recent app update. App reviews have a dynamic nature and their discussed topics change over time. The changes in the topics among collected reviews for different versions of an app can reveal important issues about the app update. A main technique in this analysis is using topic modeling algorithms. However, app reviews are short texts and it is challenging to unveil their latent topics over time. Conventional topic models such as Latent Dirichlet Allocation (LDA) and Probabilistic Latent Semantic Analysis (PLSA) suffer from the sparsity of word co-occurrence patterns while inferring topics for short texts. Furthermore, these algorithms cannot capture topics over numerous consecutive time-slices (or versions). Online topic modeling algorithms such as Online LDA (OLDA) and Online Biterm Topic Model (OBTM) speed up the inference of topic models for the texts collected in the latest time-slice by saving a fraction of data from the previous time-slice. But these algorithms do not analyze the statistical-data of all the previous time-slices, which can confer contributions to the topic distribution of the current time-slice.In this paper, we propose Adaptive Online Biterm Topic Model (AOBTM) to model topics in short texts adaptively. AOBTM alleviates the sparsity problem in short-texts and considers the statistical-data for an optimal number of previous time-slices. We also propose parallel algorithms to automatically determine the optimal number of topics and the best number of previous versions that should be considered in topic inference phase. Automatic evaluation on collections of app reviews and real-world short text datasets confirm that AOBTM can find more coherent topics and outperforms the state-of-the-art baselines. For reproducibility of the results, we open source all scripts.

The competitive market of mobile apps requires app developers to consider the users' feedback frequently. This feedback, when comes from different resources, e.g. App Stores and Twitter, will provide a broader picture of the state of the app, as the users discuss different topics on each platform. Automated tools are developed to filter the informative comments for app developers. However, to integrate the feedbacks from different platforms, one should evaluate the similarities and/or differences of the text from each one. Different meaning of the words in various context, makes this evaluation a challenging task for automated processes. For example, Request night theme and Add dark mode are two comments that are requesting the same feature. This similarity cannot be identified automatically if the semantics of the words are not embedded in the analysis. In this paper, we propose a new framework to analyze the users' feedback by embedding their semantics. As a case study, we investigate whether our approach can identify the similar/different comments from Google Play Store and Twitter, in the two well studied classes of bug reports and feature requests from literature. The initial results, validated by expert evaluation and statistical analysis, shows that this framework can automatically measure the semantic differences among users' comments in both groups. The framework can be used to build intelligent tools to integrate the users' feedback from other platforms, as well as providing ways to analyze the reviews in more detail automatically.

Distributed software systems (DSS) are usually open-ended systems used in different domains such as robotics, energy, health, etc. Multi-agent system (MAS) are a sub-class of DSS.In DSS, maintaining consistency between the system iterations is a complex and expensive task that requires coping with requirements changes and systems upgrading.The interactions, complexity and decentralized communication between components of the DSS may emerge an unwanted behavior.An unwanted behavior, known as Emergent Behavior (EB) or Implied Scenario (IS), could lead to irreversible damages.Thus, detecting IS at an early stage of the system development is needed to decrease the cost of maintaining the system.This work focuses on verification of DSS that its requirements modeled using Message Sequence Chart (MSC).The system verification focuses on the detection of IS using two already proposed different approaches.This article presents the combination of the two approaches by improving the usability of the tool presented in the first approach and the catalogue presented in the second approach.This combination allows the detection of new implied scenarios not detected using the cited approaches separately.

The paper describes a toolkit that integrates spatially-aware multi-surface systems with mixed-reality approaches to create immersive collaborative environments. The toolkit integrates multiple digital displays and multiple Microsoft HoloLens devices with multiple Kinects. The HoloLens' allow several users to look at the same virtual hologram while the Kinects enable them to use body movements to interact with these holograms as well as with other digital surfaces in a space. Effectively, the toolkit enables its users to build collaborative applications that utilize digital displays as well as the space between them to interact with information. Our approach also facilitates the management of virtual objects and overcomes gesture restrictions of HoloLens. We use the toolkit in a prototype application for improving the efficiency and response time in emergency management.

The verification of Multiagent Systems (MAS) and Distributed Software Systems (DSS) has taken a special attention due to the growing demand of having DSS in recent years. The distributed functionality and lack of having a central control in MAS and DSS may cause to emerge new behaviors in the execution time. This unexpected behavior which was not seen in the requirements is known as emergent behavior and may cause irreparable damages. Detection of these emergent behaviors is more valuable and cost effective in the early phases compared to detecting them after the deployment. In this paper we propose a new technique for the detection of a specific type of emergent behavior in the design phase. We take the advantage of social network visualization in this method. The novelty and direct advantage of this technique is presenting the exact point and cause of emergent behavior.

This research is intended to automatically detect emergent behaviors of scenario based Distributed Software Systems (DSS) in design phase. The direct significance of our work is reducing the cost of verifying DSS for unexpected behavior in execution time. Existing approaches have some drawbacks which we try to cover in our work. The main contributions are modeling the DSS components as a social network and not using behavioral modeling, detecting components with no emergent behavior, and investigating the interactions of instances of one type.

Multi agent systems with autonomous interaction, negotiation and learning capabilities can efficiently model social behavior of individuals participating in a social network. A central problem in a social network is to identify the nodes that actively participate in the expansion of the net both physically and functionally. Several metrics have already been proposed to identify those hot spots. The algorithms to identify hot spots are either heuristic based or computationally expensive. In this paper we use an agent model of the social net and propose a method that can identify the neutral nodes, i.e. the nodes that can never be considered as hot spot nodes given the network topology and rules of negotiation among nodes. Therefore these nodes can be eliminated from the net. A direct advantage of this method is reducing the computational complexity for the configuration and identification of hot spots. Through a case study we have shown that the proposed method can lead to 33% reduction of computation regarding the number of agent types in the example.

In design of distributed systems with specification languages such as message sequence charts (MSC), communication between different component (agent) types or instances of them are defined. There are a number of methods to verify the design using scenarios of inter-component communication. Those methods usually ignore the intra-component communication, i.e. communication between components of the same type. However in large scale systems, such as e-commerce systems, there are several components of one type that may communicate with each other and this may violate some regulatory policies defined in the design. On the other hand, there are declarative policies in system design that need to be integrated in the implemented system. In this paper a method that takes a topology of the system and regulatory policies as its inputs and detects the components having emergent behavior at its output is proposed. This method is defined to reveal the components that may violate the policies in the design phase by defining message types and extracting a version of MSCs called modified MSCs (MMSCs). Then by clustering and analyzing the send messages in the communications of different components the violating components are detected. By applying this method, all instances of components can be examined for policy violation in the implemented system. The method is explained along with a case study of a realistic online auction system and it is shown how this method can detect the components with emergent behaviors.