The emergence of Large Language Models (LLMs) provides a new solution to text generation tasks that involve high complexity, such as text style transfer (TST) tasks. However, previous studies have not fully explored the TST capabilities of different LLMs, and have faced issues with a lack of uniform standards in the human evaluation stage. This makes the results of human evaluation difficult to reproduce and less credible. To address this, this paper designs a prompt template to guide the cutting-edge LLMs to perform effective text style transfer and carries out an in-depth comparative analysis of various small-scale language models. In the stage of human evaluation, this paper eschews the conventional rating system, opting instead for a comparative human assessment methodology, which we refer to as duel-ranking. This method determines the relative ranking of models through mutual comparison, serving as an alternative to direct scoring. Detailed evaluation instructions are provided herein, to enhance the reproducibility of this method and ensure consistency throughout the evaluation process. This manual evaluation process reveals that GPT-3.5 and GPT-4 exhibit excellent performance in the TST tasks.

Due to the rapid growth and strong competition of mobile application (app) market, app developers should not only offer users with attractive new features, but also carefully maintain and improve existing features based on users' feedbacks. User reviews indicate a rich source of information to plan such feature maintenance activities, and it could be of great benefit for developers to evaluate and magnify the contribution of specific features to the overall success of their apps. In this study, we refer to the features that are highly correlated to app ratings as key features, and we present KEFE, a novel approach that leverages app description and user reviews to identify key features of a given app. The application of KEFE especially relies on natural language processing, deep machine learning classifier, and regression analysis technique, which involves three main steps: 1) extracting feature-describing phrases from app description; 2) matching each app feature with its relevant user reviews; and 3) building a regression model to identify features that have significant relationships with app ratings. To train and evaluate KEFE, we collect 200 app descriptions and 1,108,148 user reviews from Chinese Apple App Store. Experimental results demonstrate the effectiveness of KEFE in feature extraction, where an average F-measure of 78.13% is achieved. The key features identified are also likely to provide hints for successful app releases, as for the releases that receive higher app ratings, 70% of features improvements are related to key features.