In recent years, reinforcement learning has made significant progress in a wide range of domains, including autonomous driving, behavior analysis in electricity markets, and robotic control. Many of these studies have demonstrated the effectiveness of applying Multi-Agent Systems (MAS), where multiple agents act cooperatively or competitively. However, a major challenge in MAS lies in the increased environmental uncertainty caused by the influence of other agents, which can lead to instability in the learning process. To address this issue, various approaches have been proposed that aim to improve learning efficiency by leveraging the experience data of other agents. Previous studies have reported that sharing full experience data among agents can enhance learning efficiency in both cooperative and competitive settings. Furthermore, there are cases where sharing only partial experience data has also led to performance improvements. In this paper, we focus on the similarity between agents as a key criterion for selecting shared data and demonstrate its effectiveness in improving learning performance in MAS. Specifically, we propose a novel method that selectively shares experience data based on agent similarity and utilizes this data to complement an agent’s own experiences. Through simulations with heterogeneous (asymmetric) agents, we show that the proposed method enhances learning efficiency in multi-agent environments.

The growing integration of renewable energy sources (RESs) within electric distribution systems poses substantial challenges, such as network constraint violations and increased monetary costs. Distribution system reconfiguration is a viable solution for addressing these problems. However, due to severe network constraints, distribution system operators often face challenges in finding optimal reconfiguration solutions within practical computational time. This paper proposes a constrained multiobjective evolutionary algorithm (CMOEA) specialized for solving the reconfiguration problems based on dual-stage and dual-population strategies to tackle these difficulties. The proposed CMOEA employs specialized selection pressures for each population in each stage to guide the populations toward good objective values with diversity. The proposed CMOEA is applied to computational experiments of a reconfiguration problem characterized by severe constraints and uncertainties arising from RESs. The results demonstrate its effectiveness in identifying feasible solutions with good objective values, i.e., the low investment cost.

This work in progress resaerch-to-practice mainly contributes the development of a multistage classification method to distinguish between learners who are easily predictable and those who exhibit more complex patterns. This methodological innovation not only supports the efficient allocation of educational resources but also aids in customizing learning materials for learning management systems, enhancing the relevance and effectiveness of educational content. In Japan, the GIGA School Concept, which involves the use of one internet-connected electronic device per student, has been introduced in all elementary and junior high schools since 2020. The integration of IoT (Internet of Things) technology supported by such initiatives represents a significant shift towards creating more interactive and tailored learning environments. This evolution aims not only to enhance educational outcomes by providing individual electronic devices to learners but also to optimize learning results and operational efficiency. These technological tools enable learners to progress through their educational journey at their own pace, promoting a more learner-centered approach known as “Personalized Learning.” Simultaneously, educators can more effectively monitor and manage the learning process, enabling a more responsive and adaptive educational experience. At the central of this transformation is the role of machine learning models (such as neural networks and random forests) used to predict learners' performance based on past academic achievements. This predictive capability allows for the grouping of learners based on the predictability of their performance, further refining educational strategies to meet diverse learning needs. Furthermore, this study explores the practical application of these models through numerical experiments using actual learner data. The results highlight the potential of combining IoT technology and machine learning to revolutionize the educational domain by providing personalized learning experiences. Such an approach promises not only to improve learning outcomes by aligning educational content and strategies with individual learners' profiles but also to streamline the educational process, reduce the burden on educators, and optimize the overall educational environment. In conclusion, the integration of IoT and machine learning into education presents a visionary method for addressing the unique needs and potential of individual learners.

In current Japanese education, realizing individualized and optimal learning through the effective use of ICT terminals has been required. Adaptive learning support systems are expected to help solve the above issues. This study proposes an adaptive learning support system that maximizes the learner's characteristics such as the ability and motivation to learn using an extremum-seeking control method. The proposed system is validated by using a mathematical learner model that includes the forgetting factor of human short-term memory. The simulation result shows that the system maximized the learner's characteristics by tuning the assist ratio.

In recent years, there have been growing expectations for the improvement of operational efficiency through the use of artificial intelligence (AI: Artificial Intelligence) in various fields such as healthcare, industry, and services. Additionally, advancements in robotics technology have enabled widespread collaborative operations among multiple autonomous mobile robots. Since the 1980s, there has been active research on improving the efficiency of coordinated actions by multiple autonomous mobile robots using Multi-Agent Robot Systems (MARS) (Cao et al.; 1997, Yan et al.; 2013, Ismail and Sariff; 2019). This study focuses on robots that transport cargo within a warehouse and aims to develop learning methods for effective division of labor among multiple robots. In large and complex environments, agents need to undergo extensive repetitive learning to make appropriate action choices solely through machine learning techniques such as reinforcement learning. Traditional multi-agent methods like QMIX (Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning) (Rashid et al.; 2020) and MADDPG (Multi-Agent Deep Deterministic Policy Gradient) (Lowe et al.; 2017) require substantial computational time for environmental exploration, presenting a significant challenge. Therefore, this paper proposes a two-tiered learning model that separates overall optimization, such as establishing a general procedure for task performance, from individual optimization based on local situational judgments. Additionally, to demonstrate the effectiveness of the proposed method, a simulation system tailored to the target environment is constructed, and simulation analysis based on this system is conducted.

Particle Swarm Optimization tends to converge prematurely to local solutions. To improve such a problem, TCPSO (Two-swarm Cooperative PSO) has been proposed. TCPSO employs a group of slave particles for intensive solution search and a group of master particles for global solution search. If the solution search process of TCPSO results in a small update range of solutions, the approximate shape of the function can be estimated based on known input data. This estimation can be done using Gaussian process regression to model the distribution with mean and variance, and improves the solution search process of TCPSO.

This study presents a novel approach to enhance the performance of TCPSO by integrating Gaussian process regression into the solution search process. By leveraging Gaussian process regression to estimate the underlying function of the target problem, our proposed method aims to improve the algorithm's exploration capabilities and enable the discovery of more precise and reliable solutions. Through extensive experimentation and comparisons with existing approaches on diverse benchmark problems, we demonstrate the effectiveness and superiority of our approach in achieving improved search performance and solution quality.

This research proposes an adaptive smart learning support system, whose target is a typing support system, based on the Model-Based Development (MBD) approach. The proposed typing support system adaptively adjusts the level of work so that the typing skill of a learner is smoothly grown. In this paper, the goal of an adaptive smart learning support system is briefly explained, and a concrete design scheme of a part of the support system based on the MBD approach is presented. This paper also mentions the work in progress of developing an actual typing support system.

PSO (Particle Swarm Optimization) is attracting attention in recent years to solve the multivariate optimization problems. In PSO, multiple individuals (particles) which records its own position and velocity information are placed in the corresponding search space, and the particle swarm move to discover the optimal solution by sharing information with other particles. The search process of PSO has problem such that it is difficult to deviate from the local solution because of convergence speed of the swarms is too fast. In TCPSO (Two-Swarm Cooperative PSO), particle swarm consists of two different types of particles (a master particle swarm and a slave particle swarm) with different characteristics of search process. Experimental results of using several benchmark problems indicate that TCPSO has high performance of finding optimal solutions for multidimensional and nonlinear problems. This study introduces the concept of specificity of each master particle which indicates the diversity of master particle swarm, and proposes an algorithm that improves the efficiency of the solution search process in TCPSO by periodically analyzing the behavior of master particle swarm. This study conducts several numerical experiments for verifying the effectiveness of the proposed method.

This paper to Practice Full Paper presents about a procedure to generate the learners' data using a learner model based on first-order lag system to generate learners data. By using the generated data, the learners are classified into several groups and some learners with low understanding degree can be extracted by using the neural networks. It is necessary to provide learning support corresponding to the understanding degree of each learner in a class to improve effective learning. By providing additional education for the learners who are predicted as low degree by the proposed procedure, it is expected to take countermeasures for not becoming “dropout students” in early stage. For this purpose, predicting the understanding degree is important, and this paper employs a recurrent neural network as the predictor. Hayashida et al. (2018) have constructed to classify the learners by understanding degree at the end of the class based on some observed data such as result of quizzes, or report tasks by using FNN (Feedforward Neural Network). This paper uses a RNN (Recurrent Neural Network) which consists of feedforward signal processing and structure of the signal feedbacks because of the observed data is time-series data. This paper proposes a learner model based on first-order lag system to generate learner data for training RNNs. As experimental result of the simulation, this paper succeeds in extracting the learner group with low understanding degree in future.

The aim of this study is to predict the achievement degree of each student at the end of a lecture, based on a simple questionnaire result which regularly surveys degree of the subjective understanding conducted to students in a class. In this study, the feedforward neural networks (FNNs) and decision tree are used for the prediction and the classification. A FNN which is with a multiple input and multiple output structure is well known that it has high performance for multidimensional data prediction or classification. Therefore, FNNs are considered to be suitable for the problem dealt with in this study such that student classification based on multiple questionnaire results. Additionally, it is possible to analyze students' learning process in detail by using a decision tree that can obtain student classification rules in an explicit form. This study conducts an experiment using data of a classification six times questionnaire surveys and a final examination and constructed a system for student classification based on the the answers of three questionnaire. Experiment has succeeded in roughly classifying learners into three clusters based on achievement degree. It means that the proposed method is predict the potential comprehension degree of a student. Sequentially, by providing additional education for the students who are classified as a low degree, it is expected to be able to take countermeasures for not becoming “dropout” in early stage.

In order to carry out simulation using an environment in the real world, reinforcement learning techniques are improved to be applied on continuous environment. Real values are available as the input and output value of neural networks, however, it can not be applied to the multi step problem. Although the classifier system can preserve the value of actions and its prediction accuracy, there exists a problem when applying on continuous environment, because the bit string in the condition part should be long. In order to compensate for the disadvantages of both the neural network and the classifier system, this paper proposes N-XCS (Neural network eXtended Classifier System) in which the merits of these two methods are twisted together. Additionally, the usefulness of the proposed method is indicated based on the result of some numerical experiments.

ACSM (Hayashida et al., 2014) consists of a method of discerning the aliased states in a POMDP (Partially Observable Markov Decision Process) which is one of Markov decision process such that an agent observes local information about the environment, and choosing the appropriate action based on the internal memory and the sensory information which an agent obtains from the environment. Though ACSM achieves the highest performance in the existing methods based on classifier systems, it requires a huge number of memories for the internal memories, and spends long time for some large scaled problems. This paper improves a classifier system, ACSM (Anticipatory Classifier System with Memory) focused on the process of learning of ACSM, and the aim of this paper is to make the system more efficient. The improved method is named ACSMr in this paper, and some numerical experiments using 5 kinds of maze problems which are well used as benchmark problems for POMDPs are executed. ACSMr achieves greater experimental result than the existing classifier systems for the maze problems.

Sun and Li (2014) have proposed TCPSO(Two-swarm Cooperative Particle Swarm Optimization) that the swarms are divided into two groups with different migration rules. TCPSO has higher performance for high-dimensional nonlinear optimization problems. This study revises TCPSO to avoid inappropriate convergence of the swarms. The quite feature of the proposed method is that the population have same migration rules. However, through that the swarms are divided into some clusters based on distance measure, k-means clustering method, both diversity and centralization of search process are maintained, and it increases the potential of attainment to the global optimal solution. This study conducts numerical experiments using several types of functions, and the experimental results indicate that the proposed method has higher performance than the TCPSO for large-scale optimization problems.

This paper improves a classifier system, ACS (Anticipatory Classifier System). The suggested classifier system is named ACSM (ACS with Memory) which consists of a method of discerning the aliased states in a POMDP (Partially Observable Markov Decision Process), and choosing the proper action based on the internal memory and the sensory information around the agent. A POMDP is one of Markov decision process such that an agent observes local information about the environment. This paper executes some numerical experiments using eight kinds of maze problems which are well used as benchmark problems for POMDPs. ACSM achieves greater experimental result than the existing classifier systems for the maze problems.

The aim of this research is behavioral analysis of the human subjects in laboratory experiments of simultaneous ultimatum game through agent-based simulation. Andreoni and Blanchard (2006) conducted a laboratory experiments using human subjects, and they observed that deviant behavior of the subjects from Nash equilibrium. Similarly, in several laboratory experiments of ultimatum games in a form of sequential game, deviant behavior of the human subjects from subgame perfect equilibrium are observed. The related literature have suggested a mathematical model incorporating fairness of payoffs (Duffy and Hopkins; 1999), and adaptive models based on reinforcement learning (Duffy and Feltovich; 1999). This study constructs simulation model using adaptive agents which makes decision by trial and error approach based on neural networks and genetic algorithms. The experimental result indicates that the behavior of subjects can explained by decision mechanism by trial and error approach, interaction between human subjects, and risk attitude.

This paper provides a new evolutionary multiobjective optimization method for automatically optimizing the network topology of recurrent neural networks (NNs). To obtain NNs with higher prediction capability for time-series data, the proposed method is constructed by focusing on the intensively exploration of a feasible region including solutions with small training errors on the Pareto frontier, unlike existing evolutionary multiobjective optimization methods, which aim to find a whole set of the Pareto optimal solutions. Our method is characterized by the ideas of self-adaptive mutation probability setting, elite preservation strategies and archive for the preservation of local optimal solutions. Through the comparison with the performances of the most promising existing method by Delgado et al. using benchmark time-series data instances, it is shown that the proposed method is superior to the existing effective algorithm with respect to the capability of time-series prediction.

In this paper we focus on a multiobjective two-level 0-1 programming problem in which the decision maker at the upper level has an objective function and the decision maker at the lower level has multiple objective functions. We assume that there is not coordination between the decision maker at the upper level and the decision maker at the lower level. The decision maker at the upper level must take account of multiple rational responses of the decision maker at the lower level in the problem. We examine two kinds of situations based on anticipation of the decision maker at the upper level; an optimistic anticipation and a pessimistic anticipation. We show mathematical programming problems for obtaining the Stackelberg solutions based on two kinds of anticipation and propose computational methods using genetic algorithms for obtaining the Stackelberg solutions. In order to demonstrate feasibility and effectiveness of the proposed computational methods through genetic algorithms, we plan to conduct numerical experiments.

In some published papers of the network formation, it is assumed that each player pays the same amount of cost for forming or maintaining a link, or a player who offers to form new link pays all of the link cost. In this paper, however, we construct two types of network formation models with general allocation procedures of the link cost, and examine stability of the network. In the first model, a pair of players share the cost unequally with a fixed fraction, and in the second model, the players divide the link cost in accordance with the procedure of the ultimatum bargaining games.

With existence of the social custom or norm, Naylor demonstrates a possibility of stable long-run equilibria of support for a strike in a labor market, and this implies that at least some individuals will behave cooperatively and hence the prisoners' dilemma could be escaped. In this paper, we develop an agent-based simulation system in which artificial adaptive agents have mechanisms of decision making and learning based on neural networks and genetic algorithms, and compare the result of our simulation analysis with that of the mathematical model by Naylor. Especially, while the Naylor model is based on rationality about maximization of individual utility, our agent-based simulation model employs adaptive behavior of agents; agents make decisions by trials and errors and they learn from experiences to make better decisions.