The paper introduces LLM2FedLLM, a tool designed for Software Engineering (SE) researchers to simulate fine-tuning Large Language Models (LLMs) within a federated learning (FL) framework. Unlike existing FL frameworks that facilitate real client collaboration, our simulator provides a controlled environment for experimenting with FL scenarios on a single machine. The LLM2FedLLM Simulator addresses SE code tasks, such as code summarization, code review, and code translation, within a federated learning framework by first partitioning the selected code dataset into heterogeneous subsets for multiple clients. It then fine-tunes the chosen LLM and evaluates its performance against vanilla, centralized, and individual client models using various metrics. The tool supports several federated aggregation methods and PEFT for supervised learning, with the flexibility to easily integrate additional techniques. The evaluation of our tool on Python code summarization showed that FedLLM performs comparably to centralized models and outperforms individual clients, particularly in low-data scenarios. Our tool aims to facilitate research advances in secure collaborative training simulations within the SE community. https://youtu.be/-byKkaiBchw.

Software Engineering (SE) researchers are extensively applying Large Language Models (LLMs) to address challenges in SE tasks such as code clone detection, code summarization, and program comprehension. Despite promising results, LLMs have to be fine-tuned and customized with specific datasets for optimal performance. However, the proprietary nature of SE data, and the lack of LLMs trained on non-open source data is an open problem. While there exists work on applying Federated Learning (FL) for SE, integration of FL with LLMs for SE is unexplored. Hence, we propose a FedLLM for “code summarization” as developers spend more time in comprehending code. We setup a federated learning architecture and fine-tune LLM (Llama2 with 6.7B parameters) using Parameter Efficient Fine-Tuning (PEFT) for code summarization. We conducted our experiments on 40GB RAM GPU in an A100 architecture. Results show that FL-trained LLM is as effective as a centrally-trained one. We envision that leveraging non-open source data using FedLLM for SE could be an interesting research direction.

The developer community has witnessed an unprecedented surge in recent years, with over 100 million active developers on the GitHub platform in 2023. Along with it, there is a significant rise and adoption of new programming languages, frameworks and tools. The study aims to comprehend how developers perceive these fast-growing programming languages by performing emotion analysis of developer's comments posted in various software artifacts such as pull requests, issues and commits of GitHub repositories. In this regard, we employed a fine-tuned small 'Large Language Model' (sLLM) to detect emotions, leveraging a balanced dataset from existing literature complemented with additional manual annotations from our collected data. We have analyzed 10 fast-growing programming languages, examining 1.8 million comments from 4.1 million non-code artifacts. To further validate our findings, we have performed a qualitative survey and analysis with 28 developers. Our study reveals insights into the developers emotion associated with these fast-growing languages. Notably, "Surprise" is the predominant emotion associated with these languages.