The emergence of large language models (LLM) has given rise to a growing concern regarding the generation and dissemination of inaccurate information through these technologies. Addressing this issue requires a benchmark for the safety of LLM for Japanese. However, existing benchmarks are limited in that fail to adequately incorporate the unique falsehoods and erroneous information that are actively circulating on social media in Japan. This study proposes JSocialFact, a benchmark for evaluating the safety of LLM based on misleading information in Japan. The benchmark is created by manually annotating the data extracted from X posts and community notes capturing a wide range of misleading, false, or malicious information currently circulating on social media. Both manual and automatic evaluations using GPT-4 revealed discrepancies in how models handle harmful content. Although GPT-4’s evaluation showed some correlation with human judgment, notable discrepancies were observed, with GPT-4 frequently assigning higher safety scores than human evaluators. JSocialFact is the first dataset constructed from actual social media logs in Japanese, specifically designed to evaluate the safety of LLM outputs in addressing misinformation.

The proximity status of the individual poses a significant infection risk since infectious diseases can be transmitted through contact with others. Implementing crowd control measures to alleviate congestion is crucial in preventing infection spread during large-scale events with numerous spectators. To assess effective crowd control, comparing pedestrian behaviors and employing appropriate methods is essential. In this study, we propose a method for visualizing the influence of pedestrian behavior on infection risk by representing proximity as a network.

Appropriate evacuation guidance for specific events and facilities can be safely and rapidly achieved by controlling gates and emergency exits. However, in the case of large-scale evacuations, such as those in cities, it is difficult to obtain an optimal control method because of the complexity of the entire city. Understanding changes in pedestrian movement and urban characteristics requires an urgent need to build a dataset based on simulation results from various scenarios. In this study, we discuss the requirements for datasets in various studies, such as agent simulation, machine learning, and artificial intelligence.

Estimating individual treatment effects from observational data is a fundamental problem in causal inference. To accurately estimate treatment effects in the spatial domain, we need to address certain aspects such as how to use the spatial coordinates of covariates and treatments and how the covariates and the treatments interact spatially. We introduce a new problem of predicting treatment effects on time series outcomes from spatial graph data with a hierarchical structure. To address this problem, we propose a spatial intervention neural network (SINet) that leverages the hierarchical structure of spatial graphs to learn a rich representation of the covariates and the treatments and exploits this representation to predict a time series of treatment outcome. Using a multi-agent simulator, we synthesized a crowd movement guidance dataset and conduct experiments to estimate the conditional average treatment effect, where we considered the initial locations of the crowds as covariates, route guidance as a treatment, and number of agents reaching a goal at each time stamp as the outcome. We employed state-of-the-art spatio-temporal graph neural networks and neural network-based causal inference methods as baselines, and show that our proposed method outperformed baselines both quantitatively and qualitatively.

Because human movement spreads infection, and mobility is a good proxy for other social distancing measures, human mobility has been an important factor in the COVID19 epidemic. Therefore, the control of human mobility is one of the countermeasures used to suppress an epidemic.As a notable feature, COVID19 has had multiple waves (subepidemics). Understanding the causes of the start and end of each wave has important implications for a policy evaluation and the timely implementation of countermeasures. Some of the waves have been correlated with the changes in mobility, and some can be attributed to the emergence of new variants. However, the start and end of some of the waves are difficult to explain through known factors.To evaluate the effect of human mobility, we built a stochastic model incorporating individual movements of 500,000 people obtained from anonymized, user-approved location data of smartphones throughout Japan. Instead of using aggregate values of human mobility, our model tracks the movements of individuals and predicts the infection of all persons within the entire country. Although the model only has a single static parameter, it successfully reproduced the occurrence of three waves of the number of confirmed cases within the study period of March 01 to December 31, 2020 in Japan. It was previously difficult to explain the end of the second wave and the start of the third wave in the study period by human mobility alone. Our results suggest the importance of tracking individual movements instead of relaying the aggregate values of human mobility.