As large-scale web-based experiments become increasingly feasible, concerns have arisen regarding the reproducibility and external validity of their results. Small-scale laboratory experiments, which allow for more precise control over various attributes of participants, can complement the findings of large-scale web-based experiments and contribute to rigorously examining human behavior without bias. Specifically, this study investigates the impact of cognitive ability in simple decision-making contexts such as the Dictator and the Ultimatum Game. We find that in some situations there is no difference in the behavior of the participants regardless of the difference in Raven’s score, in other situations the different score produces different behavior. These results suggest that we should control the cognitive ability of the participants according to the situations.

It is necessary to extract some patterns of human behavior from events observed in subject experiments in order to choose particular parameters for computer experiments on trades of information. Many experimental sites are, however, faced with practical problems due to their small subject pools. In conducting a subject experiment at multiple sites for avoiding those problems, homogeneity of subjects’ behavior is important for integrating the data collected at separate sites. Ideally, preliminary test results on the homogeneity are needed in planning the experiment. This paper clarifies a condition under which subjects’ behavior in a bandit experiment with the context of weighted voting can be homogenized across different subject pools in five universities in Japan by covariate adjustment with their cognitive ability scores on Raven’s APM test. Among experimental sites located in different regions, we could not obtain homogeneity of subjects’ behavior by covariate adjustment with their cognitive ability scores in addition to their attribute information, although it was not difficult to obtain the homogeneity across the experimental sites located in the same region.

This note presents two remarks about the specifications of the subject experiment designed and conducted by [4] regarding the problem of assigning students to schools as a large-scale data processing problem. We compared the efficiency of assignments under two mechanisms the Boston mechanism (Bos) and the deferred acceptance mechanism (DA). First, we first conducted a subject experiment in which subjects were categorized into two groups according to their scores of the Raven Advanced Progressive Matrices (APM) test, which measures cognitive ability. Subjects needed to compute their priority orders for schools from a queue of numbers. Next, we input the behavioral data observed in the early rounds of the subject experiment to the subjects’ submission of their preferences for schools and generated assignments under the Bos and the DA in order to create a virtual situation where subjects believed that the priority orders were actually unchanged, although those orders had been changed. Our main observations are as follows. (1) We should control the subjects’ cognitive ability scores and the mixtures of subjects with those scores in the experimental sessions in order to obtain the same result as the one observed by [4]. (2) Subjects might not compute the priority orders of students for schools exactly. Future research should further examine the conditions necessary to reproduce the experimental results of [4], based on the remarks clarified in this note,