With the rapid expansion of large language model inference service users, cloud computing resource costs have become a critical challenge for service providers. Although utilizing end-device resources for auxiliary inference provides new possibilities to reduce cloud computing costs, existing solutions struggle to achieve an ideal balance across multi-task accuracy, end-to-end latency, and cloud computing costs.

Code completion has become an important feature of today’s integrated development environments (IDEs). This task involves predicting the next code token(s) based on its contextual information within the code. However, most existing code completion approaches do not consider users’ feedback during the completion process. In this paper, we propose a framework, EHOPE (Enhance Code Completion with Implicit Feedback)), which exploits LSTM(Long Short-Term Memory) and pre-trained model BERT(Bidirectional Encoder Representation from Transformers) to enhance the performance of token-level code completion. By leveraging users’ feedback information, we train an LSTM model to supplement the recommendation list. In addition, we re-rank the list of recommendations using the pre-trained model BERT, which is fine-tuned with feedback information. Existing token-level code completion tools can be plugged into EHOPE. We choose two representative code completion approaches from different categories: one based on statistical methods and the other based on deep learning. These approaches serve as baselines to showcase the performance improvements of EHOPE, evaluated using Hit@k (Top-k) and MRR(Mean Reciprocal Rank) metrics. Empirical experiments show that the recommendation performance steadily and substantially improves as the feedback data increases compared with the baselines.

Efficient application programming interface (API) recommendation is one of the most desired features of modern integrated development environments. A multitude of API recommendation approaches have been proposed. However, most of the currently available API recommenders do not support the effective integration of user feedback into the recommendation loop. In this paper, we present BRAID (Boosting RecommendAtion with Implicit FeeDback), a tool which leverages user feedback, and employs learning-to-rank and active learning techniques to boost recommendation performance. The implementation is based on the VSCode plugin architecture, which provides an integrated user interface. Essentially, BRAID is a general framework which can accommodate existing query-based API recommendation approaches as components. Comparative experiments with strong baselines demonstrate the efficacy of the tool. A video demonstrating the usage of BRAID can be found at https://youtu.be/naD0guvl8sE.