Sensitivity analysis is fundamental and essential in analysis and design of any system. This paper proposes a method of sensitivity analysis for rhythm phenomena in gene regulatory networks (GRNs). In particular, we focus on cyclic expression pattern sequences in GRNs and sensitivity of period which plays one of important roles in periodic phenomena. A piecewise-linear differential-equation model is utilized as a model of GRNs. Rhythm phenomena are expressed by using periodic orbits and corresponding expression pattern sequences. Sensitivity analysis of rhythm phenomena is very difficult because rhythms appear autonomously as periodic phenomena in nonlinear systems. and only a few studies have been done. In general, sensitivities of period are calculated by using numerical methods approximately. In this paper, we analytically derive the mathematical expression of period sensitivity of GRNs. It is shown through numerical examples that the proposed method makes it possible to obtain period sensitivity appropriately.

Sensitivity analysis is a basic and essential issue in analysis and design of any system. In this paper, we discuss a method of sensitivity analysis of rhythm phenomena which are found in various systems such as physical systems, biological systems and social systems and so on. Sensitivity analysis of rhythm phenomena is very difficult because rhythms appear autonomously as periodic phenomena in nonlinear dynamical systems and only a few studies have been done. We have already proposed an analysis method of period sensitivities in rhythm phenomena. This paper discusses an analysis method of phase sensitivity in rhythm phenomena. We first define phase for periodic trajectories of nonlinear systems, that is, limit cycles and derive a rigorous mathematical expression of phase sensitivities by introducing Poincaré map. Based on the expression we derive an efficient computer algorithm to calculate the phase sensitivities. It is shown that the proposed analysis method makes it possible to obtain both the period and phase sensitivities efficiently with reasonable accuracy.

Sensitivity analysis is fundamental and essential in analysis and design of any system. This paper proposes a method of sensitivity analysis of rhythm phenomena which are found in various systems. In particular, we propose an analysis method of period sensitivities in rhythm phenomena. Analysis of period sensitivities is very difficult because rhythms appear autonomously as periodic phenomena in nonlinear systems and only a few studies have been done. We already proposed an analysis method of parameter sensitivities of the periods in rhythm phenomena based on the sensitivity equation approach. In this paper, we propose a computationally efficient and accurate method to analyze the period sensitivities by the adjoint equation approach. Comparisons of the proposed method with that based on sensitivity equations are also made.

In the last decades, the reinforcement learning method has attracted a great deal of attention and many studies have been done. However, the method is basically a trial-and-error scheme and it takes much computational time to acquire optimal strategies. Furthermore, optimal strategies may not be obtained for large and complicated problems with many states. To resolve these problems we have proposed the swarm reinforcement learning method, which is developed inspired by the multi-point search optimization methods. In this paper, we propose a sophisticated swarm reinforcement learning method incorporating inverse reinforcement learning, which can improve learning speed and obtain better solutions. The proposed method is developed especially for the partially observable Markov decision processes (POMDPs). We evaluate the proposed method through experiments and compare the results with those of the conventional swarm reinforcement learning, Q-learning and Hierarchical Q-learning (HQ-learning), It is confirmed that the proposed method makes it possible to obtain better solutions with less learning time than the existing methods, especially for the problem in POMDPs.

The purpose of this paper is to propose a method for analyzing the period sensitivities and bifurcations for rhythm phenomena. The authors have already proposed a method for analyzing the sensitivities of the period with respect to parameters, an important characteristic of rhythm phenomena, and computationally efficient algorithms for this purpose. The period sensitivities of rhythm phenomena depend on bifurcations of that phenomena. In this paper, we propose a method and computation algorithms for investigating variations of the period sensitivities and bifurcations for rhythm phenomena by incorporating those algorithms. We also show an example of investigating variations of the period sensitivities and bifurcations by using the developed algorithm and demonstrate what can be revealed by it.

Sensitivity analysis is fundamental and essential in the analysis and design of any system. Sensitivities are usually defined as derivatives of system variables or system performance with respect to its parameters. This paper discusses a sensitivity analysis method for a class of cyber-physical systems. We consider the following system as a class of cyber-physical systems at the first step of the research: a hybrid system in which a continuous-time system and a discrete-time system are connected through A/D and D/A interfaces. They are represented by block diagrams in which arbitrary elements including nonlinear elements are arbitrarily connected. It is known that the sensitivity analysis based on Tellegen’s theorem has become a standard method for electric circuits. We extend Tellegen’s theorem to the hybrid system and drive a method for computing sensitivities of any signal with respect to any parameter in the system. We also propose a method to apply the proposed method to parameter optimization of the systems. In order to evaluate the performance of the proposed method, some numerical experiments are conducted. We apply the proposed method to a cyber-physical system whose parameter sensitivities can be derived analytically in order to estimate its accuracy. It is shown through numerical experiments that the proposed method can obtain sensitivities with sufficient accuracy. Furthermore, we apply it to an optimization problem of a continuous and discrete-time hybrid system. It is shown that the proposed method makes it possible to obtain optimal parameters as the hybrid system.

Sensitivity analysis is fundamental and essential in analysis and design in any system. This paper discusses a method of sensitivity analysis of rhythm phenomena which are found in various systems such as physical systems, biological systems and human societies and so on. Sensitivity analysis of rhythm phenomena is very difficult because rhythms appear autonomously as periodic phenomena in nonlinear systems and only few studies have been done. We deal with the periods and propose an analysis method of sensitivities of periods for periodic phenomena. We first derive a strict expression of period sensitivities by introducing Poincaré map. Based on the expression we derive an efficient computer algorithm to calculate period sensitivities. It is shown that the proposed analysis method makes it possible to obtain period sensitivities of not only stable periodic orbits but also unstable periodic orbits embedded in chaos attracters.

In artificial Spiking Neural Networks (SNNs) the information processing and transmission are carried out by spike trains in a manner similar to the generic biological neurons. Recently it has been reported that they are computationally more powerful than the conventional neural networks. In biological systems there are numerous examples of autonomously generated periodic activities. Several different periodic patterns are generated simultaneously in a living body. It is known that in biological systems there are specific neurons which generate such periodic patterns. This paper presents a method for synthesis of neural oscillators by spiking neural networks. We propose a learning method for synthesizing spiking neural networks which generate desired periodic spike trains with specified spike emission times. We also propose a method for making the periodic trajectory generated by the synthesized spiking neural oscillator asymptotically stable.

Recently, models of neural networks in the real domain have been extended into the high dimensional domain such as the complex number and quaternion domain, and several high-dimensional models have been proposed. These extensions are generalized by introducing Clifford algebra (geometric algebra). In this paper we extend conventional real-valued Hopfield-type neural networks into the octonion domain and discuss their dynamics. The octonions represent a particular extension of the quaternions which also represent a particular extension of the complex numbers and have 7 imaginary parts. They are non-commutative and non-associative on multiplication and do not belong to Clifford algebra due to the latter fact. With this in mind we propose four models of octonion Hopfield-type neural networks. We derive existence conditions of an energy function and construct energy function for each model.

Recently it has been reported that artificial spiking neural networks (SNNs) are computationally more powerful than the conventional neural networks. In biological neural networks of living organisms, various firing patterns of nerve cells have been observed, typical examples of which are burst firings and periodic firings. In this paper we propose a learning method which can realize various firing patterns for recurrent SNNs (RSSNs). We have already proposed learning methods of RSNNs in which the learning problem is formulated such that the number of spikes emitted by a neuron and their firing instants coincide with given desired ones. In this paper, in addition to that, we consider several desired properties of a target RSNN and proposes cost functions for realizing them. Since the proposed cost functions are not differentiable with respect to the learning parameters, we propose a learning method based on the particle swarm optimization.

In this paper, we first extend the Wirtinger derivative which is defined for complex functions to hyperbolic functions, and derive the hyperbolic gradient operator yielding the steepest descent direction by using it. Next, we derive the hyperbolic backpropagation learning algorithms for some multilayered hyperbolic neural networks (NNs) using the hyperbolic gradient operator. It is shown that the use of the Wirtinger derivative reduces the effort necessary for the derivation of the learning algorithms by half, simplifies the representation of the learning algorithms, and makes their computer programs easier to code. In addition, we discuss the differences between the derived Hyperbolic-BP rules and the complex-valued backpropagation learning rule (Complex-BP). Finally, we make some experiments with the derived learning algorithms. As a result, we find that the convergence rates of the Hyperbolic-BP learning algorithms are high even if the fully activation functions are used, and discover that the Hyperbolic-BP learning algorithm for the hyperbolic NN with the split-type hyperbolic activation function has an ability to learn hyperbolic rotation as its inherent property.

Recently, models of neural networks in the real domain have been extended into the high dimensional domain such as the complex number and quaternion domain, and several high-dimensional models have been proposed. These extensions are generalized by introducing Clifford algebra (geometric algebra). In this paper we extend conventional real-valued models of recurrent neural networks into the octonion domain and discuss their dynamics. The octonions represent a particular extension of the quaternions which also represent a particular extension of the complex numbers, They have 7 imaginary parts and do not belong to Clifford algebra. We present a model of fully connected recurrent neural networks, which are extensions of the real-valued Hopfield type neural networks to the octonion domain. We study dynamics of the models from the point view of existence conditions of an energy function. We derive existence conditions of an energy function for the Hopfield type octonion neural networks.

In this paper, we treat a multi-robot formation problem. In this problem, multiple robots move from their respective initial positions, and they achieve to make a given target formation by reaching goal positions. The goal positions which the robots reach must be different from each other. They learn their respective goal positions and the shortest routes to the goal positions. For solving the formation problem, we recently proposed a swarm reinforcement learning method. In the method, multiple sets of the robots and environments, which are called learning worlds, are prepared and the robots in each learning world learn not only by performing a usual reinforcement learning method but also by exchanging information among the learning worlds. The method, however, sometimes fails to find not only an optimal policy but also a policy which enables the robots to make the target formation, especially in the case where the target formation is large and complicated. In order to resolve this problem, this paper proposes swarm reinforcement learning methods in which the robots learn through always taking actions according to the current policy in learning by the usual reinforcement learning method. The performance of the proposed methods is evaluated through simulated experiments.

We recently proposed a multi-objective reinforcement learning method for acquiring all Pareto optimal policies simultaneously by introducing the concept of convex hulls into Q-learning method. In this method, state-action value vectors are obtained through learning only once, and then each Pareto optimal policy is derived through scalarizing the obtained state-action value vectors by using a weight vector. The method does not require learning more than once, and finds all the Pareto optimal policies by determining weight vectors adequately and by giving them in scalarizing the obtained state-action value vectors. This paper proposes a method of determining the scalarization weight vectors. The performance of the proposed method is evaluated through numerical experiments.

In this paper, we treat a multi-robot formation problem in which each of multiple robots selects one of goal positions adequately and finds the optimal route to the goal position, and we propose a swarm reinforcement learning method for acquiring the optimal policy in the problem. In the proposed method, multiple sets of the robots and an environment, which are called learning worlds, are prepared and the robots in each learning world learn not only by performing a usual reinforcement learning method but also by exchanging information among learning worlds. The performance of the proposed method is evaluated through numerical experiments.

The problem of recovering shape from shading is important in computer vision and robotics and many studies have been done. We have already proposed a versatile method of solving the problem by model inclusive learning of neural networks. In the proposed learning method, the image-formation model is included in the learning loop of neural networks. The method is versatile in the sense that it can solve the problem in various circumstances. Almost all of the methods proposed so far assume that surface reflection properties for a target object are known a priori. It is, however, very difficult to obtain those properties exactly. In this paper we propose a method to resolve this problem by extending our previous method. The proposed method is a model inclusive learning of neural networks which makes it possible to recover shape and estimate reflection parameters of an object simultaneously. The performance of the proposed method is demonstrated through experiments.

Recently, synthesis of gene regulatory networks having desired behavior has become of interest to many researchers and several studies have been done. There exist periodic phenomena in cells and these periodic phenomena are considered to be generated by gene regulatory networks. We already proposed a synthesis method of gene regulatory networks having desired cyclic expression pattern sequences. In this paper, we propose a synthesis method for realizing not only desired cyclic expression pattern sequences but also desired periods. In the proposed method we derive a representation of periods and introduce Poincare map for realizing periodic solution trajectories with desired periods. We also introduce a discrete-time network which represent transition of expression pattern. In the problem, gene regulatory network model is given by differential equations. However, in order to synthesize gene regulatory networks we solve only the discrete-time network. Therefore desired behavior are realized efficiently. Numerical experiments are carried out to illustrate the performance of the proposed method.

This paper studies multi-objective reinforcement learning problems in which an agent gains multiple rewards. In ordinary multi-objective reinforcement learning methods, only a single Pareto optimal policy is acquired by the scalarizing method which uses the weighted sum of the reward vector, and therefore different Pareto optimal policies are acquired by changing the weight vector and by performing the methods again. On the other hand, a method in which all Pareto optimal policies are acquired simultaneously is proposed for problems whose environment model is known. By using the idea of the method, we propose a method that acquires all Pareto optimal policies simultaneously for the multi-objective reinforcement learning problems whose environment model is unknown. Furthermore, we show theoretically and experimentally that the proposed method can find the Pareto optimal policies.

Recently, models of neural networks in the real domain have been extended into the high dimensional domain such as the complex and quaternion domain, and several high-dimensional models have been proposed. These extensions are generalized by introducing Clifford algebra (geometric algebra). In this paper we extend conventional real-valued models of recurrent neural networks into the domain defined by Clifford algebra and discuss their dynamics. Since geometric product is non-commutative, some different models can be considered. We propose three models of fully connected recurrent neural networks, which are extensions of the real-valued Hopfield type neural networks to the domain defined by Clifford algebra. We also study dynamics of the proposed models from the point view of existence conditions of an energy function. We discuss existence conditions of an energy function for two classes of the Hopfield type Clifford neural networks.

The problem of recovering shape from shading is important in computer vision and robotics. In this paper, we propose a versatile method of solving the problem by neural networks. We introduce a mathematical model, which we call `image-formation model', expressing the process that the image is formed from an object surface. We formulate the problem as a model inclusive learning problem of neural networks and propose a method to solve it. In the proposed learning method, the image-formation model is included in the learning loop of neural networks. The proposed method is versatile in the sense that it can solve the problem in various circumstances. The effectiveness of the proposed method is shown through experiments performed in various circumstances.

We recently proposed swarm reinforcement learning methods in which multiple sets of an agent and an environment are prepared and the agents learn not only by individually performing a usual reinforcement learning method but also by exchanging information among them. Q-learning method has been used as the individual learning in the methods, and they have been applied to a problem with discrete state-action space. In the real world, however, there are many problems which are formulated as ones with continuous state-action space. This paper proposes swarm reinforcement learning methods based on an actor-critic method in order to acquire optimal policies rapidly for problems with continuous state-action space. The proposed methods are applied to a biped robot control problem, and their performance is examined through numerical experiments.

In artificial Spiking Neural Networks (SNNs) the information processing and transmission are carried out by spike trains in a manner similar to the generic biological neurons. Recently it has been reported that they are computationally more powerful than the conventional neural networks. It is strongly desired to derive efficient learning methods of SNNs. It is, however, much more difficult to analyze and design SNNs due to their intricately discontinuous and implicit nonlinear mechanisms. In this paper, we discuss learning methods of recurrent SNNs constructed with integrate-and-fire type spiking neurons. We have already proposed a learning method of recurrent SNNs such that they possess the desired spike trains with the specified spike emission times. The method is derived based on the sensitivity equations approach. In the learning method, however, computational time required for learning increases extremely as the number of neurons in SNNs increases. In this paper, we propose two efficient learning methods of recurrent SNNs based on the adjoint equations approach. We compare the proposed and existing learning methods from the viewpoint of computational time. It is shown that the proposed methods drastically reduce computational time.

In ordinary reinforcement learning methods, a single agent learns to achieve a goal through many episodes. Since the agent essentially learns by trial and error, it takes much computation time to acquire an optimal policy especially for complicated learning problems. Meanwhile, for optimization problems, population-based methods such as particle swarm optimization have been recognized that they are able to find rapidly the global optimal solution for multi-modal functions with wide solution space. We recently proposed swarm reinforcement learning methods in which multiple agents are prepared and they learn through not only their respective experiences but also exchanging information among them. In these methods, it is important how to design a method of exchanging the information. In this paper, we propose a swarm reinforcement learning method based on ant colony optimization, which is an optimization method inspired from behavior of real ants using trail pheromones, in order to acquire the optimal policy rapidly even for complicated reinforcement learning problems. In the proposed method, the agents exchange their information through Pheromone-Q values which we define so as to make them play the same role as the trail pheromones. The proposed method is applied to shortest path problems, and its performance is demonstrated through numerical experiments.

Associative memories are one of the popular applications of neural networks and several studies on their extension to the complex domain have been done. Associative memories should recall memory patterns, and their dynamics are greatly affected by activation functions and connection weights. The theoretical analysis on qualitative properties of neural networks is very important to associative memories. We already proposed some models of complex valued associative memory using nonlinear bounded complex functions, which are not analytic. In this paper, we present several models of orthogonal type and auto-correlation type associative memories using several nonlinear complex functions which include analytic and non-analytic functions, and investigate their behavior as associative memories theoretically. Comparisons are made among these models in terms of dynamics. Simulation studies are also done to investigate dynamics of an associative memory with singular points.

In ordinary reinforcement learning algorithms, a single agent learns to achieve a goal through many episodes. If a learning problem is complicated, it may take much computation time to acquire the optimal policy. Meanwhile, for optimization problems, population-based methods such as particle swarm optimization have been recognized that they are able to find rapidly the global optimal solution for multi-modal functions with wide solution space. We recently proposed reinforcement learning algorithms in which multiple agents are prepared and they learn through not only their respective experiences but also exchanging information among them. In these algorithms, it is important how to design a method of exchanging the information. This paper proposes some methods of exchanging the information based on the update equations of particle swarm optimization. The proposed algorithms using these methods are applied to a shortest path problem, and their performance is compared through numerical experiments.

This paper presents neural network models for identification and realization of a class of discrete event systems (DESs). We consider a class of DESs which is modeled by using finite state automata. Two neural network models are presented: one is a class of recurrent neural networks and the other is a class of recurrent high-order neural networks. The models are capable of representing the DESs with the network size being smaller than the existing models. We also discuss identification and realization methods of the DESs from a given set of input and output data by training the neural networks. Comparisons are made among the models in terms of abilities of identification and realization of the DESs.

In the biological systems there are numerous examples of autonomously generated periodic activities. Several different periodic patterns are generated simultaneously in a living body. It is known that in biological systems there are specific neurons which generate such periodic patterns. In spiking neural networks the information processing is carried out by spike trains in a manner similar to the generic biological neurons. This paper presents a method for synthesis of neural oscillators by spiking neural networks. We propose a learning method for synthesizing spiking neural networks which generate desired periodic spike trains with specified spike emission times. A method of stability analysis of the generated periodic spike trains is also discussed.

In the biological systems there are numerous examples of autonomously generated periodic activities. Several different periodic patterns are generated simultaneously in a living body. This paper discusses a method of generating periodic oscillatory trajectories in an artificial neural network. We propose a synthesis method of a neural network which generates desired autonomous periodic trajectories. The proposed method makes it possible to generate not only one periodic trajectory but also multiple different trajectories in a neural network simultaneously and to make each periodic trajectory possess specified stability degree. It is known that stability and stability degree of periodic trajectories (limit cycles) can be estimated by checking eigenvalues of Jacobian matrix of the Poincaré map defined on them. We propose a learning method of neural networks which makes the Poincaré map of each generated periodic trajectory possess specified stable eigenvalues. Experimental examples are presented to demonstrate the applicability and performance of the proposed method.

Associative memories are one of the popular applications of neural networks and several studies on their extension to the complex domain have been done. One of the important factors to characterize behavior of a complex-valued neural network is its activation function which is a nonlinear complex function. We have already proposed a model of self-correlation type associative memories using complex-valued neural networks with one of the most commonly used activation functions. In this paper, we propose two additional models using different nonlinear complex functions and investigate their behavior as associative memories theoretically. Comparisons are also made among these three models in terms of their performance: dynamics and storage capabilities.

In the biological systems there are numerous examples of autonomously generated periodic activities. Several different periodic patterns are generated simultaneously in one living body. This paper discusses a problem of generating periodic oscillatory trajectories in an artificial neural network. We propose a learning method of a neural network such that it possesses desired autonomous periodic trajectories. Especially a method to generate not only one periodic trajectory but also two or more different trajectories simultaneously at specified positions in the state space of a neural network. For this purpose we utilize a class of neural network, recurrent hybrid neural networks and develop efficient learning methods for them. Experimental examples are also presented to demonstrate the applicability and performance of the proposed method.

Dynamics of 2 types of complex-valued neural network is numerically analyzed. In [Kuroe, Y, et al., 2003], some mathematical properties of a mutually connected Hopfield-type network of nonrotating complex-valued neurons were shown, and the sufficient conditions of the existence of an energy function are derived for 2 types (types A and B) of the activation function of the neuron. In this paper, we consider the Hopfield network of rotating complex-valued neurons. The network dynamics is decomposed into the dynamics of the amplitude and phase of each neuron. For type B network, the dynamics of the phases is shown to be the dynamics of a coupled system of rotating phase oscillators with a pair-wise sinusoidal phase-difference interaction [Nishikawa, I and Kuroe, Y, 2004]. Therefore a phase synchronization, which is well known in a phase oscillator system [Kuramoto, Y, 1984], is expected also in type B network. At the same time in this type B network, the network dynamics of homogeneously rotating neurons can be transformed into the network dynamics of non-rotating neurons, for which the existence condition of an energy function is derived explicitly. On the other hand for type A network, there is no such correspondence to a phase oscillator system, nor equivalence to the network whose convergence is assured by the existence of an energy function. One recent result on a phase oscillator system is the effectiveness for an area-wide signal control of an urban traffic network. Therefore in this paper, the dynamics of type A and B complex-valued rotating neural networks is numerically investigated, especially from the point of view of the effective control of the signal offset. The similarity and the difference between the 2 types of dynamics are shown through computer simulations using a microscopic traffic simulator on several traffic flow patterns and conditions.

Complex-valued Hopfield networks which possess the energy function are analyzed. The dynamics of the network with certain forms of an activation function is de-composable into the dynamics of the amplitude and phase of each neuron. Then the phase dynamics is described as a coupled system of phase oscillators with a pair-wise sinusoidal interaction. Therefore its phase synchronization mechanism is useful for the area-wide offset control of the traffic signals. The computer simulations show the effectiveness under the various traffic conditions.

Recently models of neural networks that can directly deal with complex numbers, complex-valued neural networks, have been proposed and several studies on their abilities of information processing have been done. Furthermore models of neural networks that can deal with quaternion numbers, which is the extension of complex numbers, have also been proposed. However they are all multilayer quaternion neural networks. This paper proposes models of fully connected recurrent quaternion neural networks, Hopfield-type quaternion neural networks. Since quaternion numbers are non-commutative on multiplication, some different models can be considered. We investigate dynamics of these proposed models from the point of view of the existence of an energy function and derive their conditions for existence.

In the biological systems there are numerous examples of autonomously generated periodic activities. This paper proposes a synthesis method of neural oscillators by neural network learning. The problem is formulated as determining the weights of the synaptic connections of neural networks such that, the neural networks generate desired autonomous limit cycles. We introduce a new architecture of neural networks, hybrid recurrent neural networks, in order to enhance the capability of implementing neural oscillators. In order to generate autonomous limit cycles in the neural networks we make use of the bifurcation theory. Efficient learning methods for synthesizing neural oscillators with desired limit cycles are derived. Synthesis examples are also presented to demonstrate the applicability and performance of the proposed method.

The paper proposes a learning method for synthesizing associative memory in neural networks. The problem is formulated as determining the weights of the synaptic connections of neural networks such that, for any given set of desired memory vectors, each memory vector becomes an asymptotically stable equilibrium point of the network. We introduce a new architecture of neural networks, hybrid recurrent neural networks, in order to enhance the capability of implementing associative memories. An efficient learning method for synthesizing associative memories is proposed. The proposed method assures that all the memory vectors become asymptotically stable equilibrium points with the prescribed degree of stability. Synthesis examples are presented to demonstrate the applicability and performance of the proposed method.