Existing code generation benchmarks, such as HumanEval, primarily evaluate universal algorithmic problems that do not address region-localized requirements. In realworld software development, localized knowledge, such as legal systems, cultural practices, and local language-specific processing is often required. To address this gap, we propose SakuraEval, the first region-localized code generation benchmark designed to evaluate Large Language Models, (LLMs,) performance in generating code that requires Japanese-localized knowledge. As a demonstration, of its effectiveness, we evaluated five LLMs with different language backgrounds using SakuraEval alongside HumanEval and JHumanEval. The results showed that all models exhibited significantly different tendencies in SakuraEval compared to existing benchmarks, with notable changes in their rankings that highlight each model’s Japanese-localized capabilities. SakuraEval provides valuable insights into the development of region-aware LLMs and underscores the importance of localized knowledge in code generation tasks.

Pre-trained language models have achieved many prominent results in natural language processing. Since software engineering widely includes many natural language documents, the application of pre-trained language models have received much attention in software engineering tasks. However, pretraining a large volume of source code requires a huge amount of computational resources and time. In this study, we propose an additional pre-training approach to a well-trained language model. Our initial results on mT5, multilingual T5 with an additional pretraining of Python code shows improved performance on multiple software engineering tasks including code generation, code summarisation, code repair, and error diagnosis.

Many infection preventing measures have been taken in COVID-19 situation. In particular, the approach called Three Cs avoidance is drawing attention. Three Cs means closed spaces, crowded places, and close-contact settings. This approach is taken in many business scenes regardless of individual situation. Such Three Cs are effective, but it is difficult for humans to always be aware of them. Various detection systems have been proposed to help understanding Three Cs situation. Most of them use cameras, CO2 sensors and so on. However, such system is costly due to introduce new equipment. Therefore, we propose a method for detecting Three Cs using only the existing widely used Wi-Fi equipment. Our method introduces unique detection parameters and performs statistical evaluation. As a result of constructing the system and evaluating it, we confirmed that it was possible to detect Three Cs with an accuracy of 86% or more.