This study proposes methods to enable neuro-symbolic artificial intelligence (AI) to learn context dependency through self-supervised learning with large language models (LLMs). While neuro-symbolic AI can utilize extensive and ambiguous knowledge bases to prove theorems, LLMs encounter significant challenges in this domain. However, conventional neuro-symbolic AI faces practical issues, as the search space for reasoning expands rapidly with the increasing size of the knowledge base. Therefore, this study utilizes self-supervised learning with LLMs to incorporate context dependency into neuro-symbolic AI, thereby reducing the search space for reasoning. Existing approaches cannot achieve this because they fail to learn context dependency. The proposed network is the first to combine neuro-symbolic AI and LLMs to reduce the search space for reasoning. The authors trained and evaluated the proposed network using a novel reasoning dataset with context and developed an evaluation metric to validate the effectiveness of search space reduction. Results demonstrate that the proposed network significantly reduces the search space while maintaining accuracy compared to existing baseline methods. This network provides a solution to the frame problem and has potential applications in laboratory automation and autonomous robotics, where reasoning with a large search space is required.

Unification is indispensable for inferences in symbolic processing. The authors propose a neural network-based solution to perform efficient unification. Symbolic processing in conventional artificial intelligence has strong inference abilities but is not well suited for handling large amounts of ambiguous data. Contrastingly, neural networks can easily handle large amounts of ambiguous data but are not well suited for making complex inferences. Therefore, the authors realized the unification of the knowledge base, including ambiguous data, using a network that combines a memory network and recurrent neural network. The novelty of the proposed network is that matching, which is a unification process, is highly efficient and substitution, which is a unification process, is robust. The proposed network enables highly efficient matching by grouping multiple terms and processing them in a memory network. Furthermore, it can handle unknown words even during substitution because it uses a recurrent neural network to perform substitution. The experimental results show that the proposed network can achieve more efficient unification of ambiguous data than the baseline. This study combines symbolic processing and deep learning and suggests that it contributes to the realization of complex inferences from large amounts of ambiguous data, which has proven challenging in conventional research. Furthermore, the use of unification, which handles large amounts of ambiguous data, facilitates the development of inference systems with human-interactive interfaces. This allows humans to obtain inference results without knowing the representations of the knowledge base.

Knowledge graphs (KGs) are useful information sources to make machine learning efficient with human knowledge. Since KGs are often incomplete, KG completion has become an important problem to complete missing facts in KGs. Whereas most of the KG completion methods are conducted on a single KG, multiple KGs can be effective to enrich embedding space for KG completion. However, most of the recent studies have concentrated on entity alignment prediction and ignored KG-invariant semantics in multiple KGs that can improve the completion performance. In this paper, we propose a new multiple KG embedding method composed of intra-KG and inter-KG regularization to introduce KG-invariant semantics into KG embedding space using aligned entities between related KGs. The intra-KG regularization adjusts local distance between aligned and not-aligned entities using contrastive loss, while the inter-KG regularization globally correlates aligned entity embeddings between KGs using multi-view loss. Our experimental results demonstrate that our proposed method combining both regularization terms largely outperforms existing baselines in the KG completion task.

The character vocabulary can be very large in non-alphabetic languages such as Chinese and Japanese, which makes neural network models huge to process such languages. We explored a model for sentiment classification that takes the embeddings of the radicals of the Chinese characters, i.e, hanzi of Chinese and kanji of Japanese. Our model is composed of a CNN word feature encoder and a bi-directional RNN document feature encoder. The results achieved are on par with the character embedding-based models, and close to the state-of-the-art word embedding-based models, with 90% smaller vocabulary, and at least 13% and 80% fewer parameters than the character embedding-based models and word embedding-based models respectively. The results suggest that the radical embeddingbased approach is cost-effective for machine learning on Chinese and Japanese.

Though there are some works on improving distributed word representations using lexicons, the improper over-fitting of the words that have multiple meanings is a remaining issue deteriorating the learning when lexicons are used, which needs to be solved. An alternative method is to allocate a vector per sense instead of a vector per word. However, the word representations estimated in the former way are not as easy to use as the latter one. Our previous work uses a probabilistic method to alleviate the overfitting, but it is not robust with a small corpus. In this paper, we propose a new neural network to estimate distributed word representations using a lexicon and a corpus. We add a lexicon layer in the continuous bag-of-words model and a threshold node after the output of the lexicon layer. The threshold rejects the unreliable outputs of the lexicon layer that are less likely to be the same with their inputs. In this way, it alleviates the overfitting of the polysemous words. The proposed neural network can be trained using negative sampling, which maximizing the log probabilities of target words given the context words, by distinguishing the target words from random noises. We compare the proposed neural network with the continuous bag-of-words model, the other works improving it, and the previous works estimating distributed word representations using both a lexicon and a corpus. The experimental results show that the proposed neural network is more efficient and balanced for both semantic tasks and syntactic tasks than the previous works, and robust to the size of the corpus.

In this paper, a natural language neural network model based on the analysis of the structure of sentences is proposed. The proposed neural network consists of 5 layers: sentence-layer, clause-layer, phrase-layer, word-layer, and concept-layer. The input text is split into different levels as sentences, clauses, phrases and words. Then neurons are allocated for each sentence, clause, phrase and word in the corresponding layers. The neurons in each of the upper 4 layers are connected to the other neurons in the adjacent layers according to the breakdown structure of each sentence in the input text. Concept-layer contains neurons of synsets. Each neuron of a synset is connected to its hypernyms, hyponyms and holonyms. Each neuron in the word-layer is connected to the neuron of its corresponding synset. Energy propagation is used to train the neural network and recall. Experiments to evaluate the association ability and the noise tolerance are performed. The results show that the proposed neural network has a fairly splendid recall ability and noise tolerance. This neural network is also applied to answer some TOEIC test questions in the reading comprehension part and achieved scores equivalent to the average level of human examinees, which shows its ability of learning knowledge in the test passages. The proposed neural network supports a novel way for artificial intelligence to flexibly learn and recall knowledge in English.

Restricted Boltzmann machine associative memory (RBMAM) is proposed in this paper. RBMAM memorizes patterns using contrastive divergence learning procedure. It recalls by calculating the reconstruction of pattern using conditional probability. In order to examine the performance of the proposed RBMAM, extensive computer simulations have been carried out. As the result, it has shown that the performance of RBMAM is overwhelming compared with the conventional neural network associative memories. For example as for storage capacity, RBMAM can store about from 2Nhiddento ANhideenpatterns, where Nhiddendenotes the number of neurons in the hidden layer. Similarly we have obtained superior performance of RBMAM in respect of noise tolerance and pattern complement.

It is important to identify the preferred features of sentences in different contexts. In particular, many people are increasingly using communication tools such as SNS and it is becoming easier to obtain user data. In this study, we estimate the preference for sentences in Facebook. More specifically, we try to improve the estimation of the importance of words in actual sentences. We conducted evaluation experiments, which showed that 1) inverse document frequency (IDF) is effective for both English and Japanese, 2) the estimated and actual preference levels were correlated.

We propose a sentence generation method that describes images. We do not use image processing technique in our proposed method. Human annotated image tags are used as image information to generate sentence. By using human annotated tags, we think this enables to describe image more relevant and user specific. Our method uses Kyoto University's case frame data and Google N-gram to generate candidate sentences. We extend these candidates to describe images more relevant. To be more precise, we added segments with missing semantic role, and added modification segments. To select one output sentence, we used fuzzy rules to grade naturalness of candidate sentences. To grading image relevance of the sentence, we scored word similarity for each word. The performance of the proposed system has been evaluated by subjective experiments and obtained satisfactory results.

Echo State Networks (ESNs) are an alternative to fully trained Recurrent Neural Networks (RNNs) showing State of the Art performance when applied to time series prediction. However, they have seldom been applied to abstract tasks and in the case of language modeling they require a number of units far superior to traditional RNNs in order to achieve similar performance. In this paper we propose a novel architecture by extending a conventional Echo State Network with a pre-recurrent feature layer and a nonlinear readout. The features are learned in a supervised way using a computationally cheap version of gradient descent and automatically capture grammatical similarity between words. They modify the dynamic of the network in a way that allows it to significantly outperform an ESN alone. The addition of a nonlinear readout is also investigated making the global system similar to a feed forward network with a memory layer.

This paper proposes a novel neural network to treat natural language. Most of the conventional neural networks can only process sentences consisted of a few words, and their applications are very simple such as metaphor understanding. The proposed network can process many complicated sentences and can be used as an associative memory and a question-answering system. The proposed network is composed of 3 layers and one network: Sentence Layer, Knowledge Layer, Deep Case Layer and Dictionary Network. The input sentences are divided into knowledge units and stored in the Knowledge Layer. The Deep Case Layer play an important role to process the knowledge units properly. The Dictionary Network also plays an important role as a knowledge based. We have carried out several experiments and they have shown that the proposed neural network has superior performances as an associative memory and a question-answering system. Especially as a question-answering system, the performance is very close to the elaborated system based on artificial intelligence.

One of the practical targets of neural network research is to enable conversation ability with humans. This paper proposes a novel natural language generation method using automatically constructed lexical resources. In the proposed method, two lexical resources are employed: Kyoto University's case frame data and Google N-gram data. Word frequency in case frame can be regarded to be obtained by Hebb's learning rule. The co-occurence frequency of Google N-gram can be considered to be gained by an associative memory. The proposed method uses words as an input. It generates a sentence from case frames, using Google N-gram as to consider co-occurrence frequency between words. We only use lexical resources which are constructed automatically. Therefore the proposed method has high coverage compared to the other methods using manually constructed templates. We carried out experiments to examine the quality of generated sentences and obtained satisfactory results.

In this paper, we propose a novel neural network which can learn knowledge from natural language documents and can perform analogy. The conventional neural networks can use only the information the networks learned: knowledge acquisition has been a serious problem. The proposed network solves it by using a large scale dictionary named Google N-gram. In the preprocessing, natural language documents are analyzed by a Japanese dependency structure analyzer named Cabocha. The results are used in the network connection learning. In the analogy process, firing patterns of neurons are memorized in memory parts. When a similar firing pattern is appeared, a memorized pattern is retrieved. This process enables analogical inference. Three kinds of experiments were carried out using goo encyclopedia and Wikipedia as knowledge source. Superior performance of the proposed neural network has been confirmed.

There are various kinds of evolutionary computations (ECs) and they have their own merits and demerits. For example, PSO (Particle Swarm Optimization) shows high ability during initial period in general, whereas DE (Differential Evolution) shows high ability especially in the latter period in search to find more accurate solutions. This paper proposes a novel and integrated framework to effectively combine the merits of several evolutionary computations. There are five distinctive features in the proposed framework. 1) There are several individual pools, and each pool corresponds to one EC. 2) Parents do not necessarily belong to the same EC: for example, a GA type individual can be a spouse of a PSO type individual. 3) Each incorporated EC has its own evaluated value (EV), and it changes according to the best fitness value at each generation. 4) The number of individuals in each EC changes according to the EV. 5) All of the individuals have their own lifetime to avoid premature convergence; when an individual meets lifetime, the individual reselect EC, and the probability of each EC to be selected depends on the EV. In the proposed framework, therefore, more individuals are allotted to the ECs which show higher performance than the other at each generation: effective usage of individuals is enabled. In this way, this framework can make use of merits of incorporated ECs. Original GA, original PSO and original DE are used to construct a simple proposed framework-based system. We carried out experiments using well-known benchmark functions. The results show that the new system outperformed there incorporated ECs in 9 functions out of 13 functions.

In this paper, we propose a 3D character creation system using an extraction method of kansei (sensibility) rule with fitness value. In the proposed system, a 3D character reflected kansei of the user is expressed with some constitution attributes. The attributes that are necessary to reflect kansei of the user are extracted as if-then rules by kansei rule extraction method. The consequent part of kansei rule has the fitness. By introducing the fitness, extracted kansei rules have priority. Therefore, kansei rules can be used effectively. 3D characters created by the proposed system and the user's evaluation values are stored as data, and kansei rules are extracted by analyzing the data. The extracted kansei rules are applied to create 3D characters. We have confirmed that the proposed system can create 3D characters reflected kansei of the user through experiments.

Visualization of geographic log-data is one of the key issues on geovisualization, which is defined as a research field of visualizing geographic information. This paper aims to visualize them interactively using graphics like thermograph, mashuped with interactive mapping system (IMS), such as Google Map. While conventional researches employ probability density function estimation algorithms, the problems are twofold. One is that the focused data should be analyzed rapidly online during the interaction between systems and users, for the map size and location can be changed flexibly with IMS. The other is that focused data may be sparse when the map is zoomed in. In general, EM algorithm, a commonly-used probabilistic density approximator, is not robust to sparseness and it takes long time for model construction. Parzen window is also a simple, well-known technique but it requires many kernels that make calculation costs high. The proposed method is a novel, simple kernel density estimator which is fast for model construction with high robustness to sparse data. The proposed method is based on Parzen window and employs a clustering algorithm inspired by fuzzy ART (Adaptive Resonance Theory) to reduce kernels. From the experimental results, estimation accuracy excels the conventional methods with various benchmarking models.

In this paper, we propose a method of mining KANSEI fuzzy rules from photos on the Internet. KANSEI is a Japanese word which means human impressions and KANSEI engineering is a method for translating KANSEI into product parameters through the analysis of data. In KANSEI engineering, some quantitative data are required for analyzing KANSEI and conventional approach uses questionnaire to collect quantitative data. However, questionnaire tends to become strained for subjects in order to collect enough data. The proposed method is an improved version of the method which authors have proposed. With the conventional system, fuzzy rules of KANSEI can be extracted from some questionnaire data and the system also requires heavy questionnaire surveys. The main purpose of the proposed method is to enable the method to work without questionnaire data by using photo data and tags on the Internet. By preparing learning data from the Internet and improving algorithms, the proposed method can extract fuzzy rules of correspondence of colors to impressions. The experimental results show that the proposed method worked efficiently.

In this paper, we propose a neural-network-based visualization system of geographic tendency. In general, there are some needs of understanding statistical data of geographic tendency, such as geographic marketing data, real-estate prices, and so on. The main purpose of the proposal is to visualize the tendency of these data online with interactive mapping systems, such as Google Maps. The proposed system generates translucent images of a heatmap, which shows the geographic tendency like thermograph. It consists of two steps: (1) construction of a tendency model with a neural network, (2) determine the color scale for the output heatmap. As for (1), a general regression neural network (GRNN) is employed to compose a tendency model by function approximation. As for (2), the output color scale is optimized and the heatmap is finally generated using the composed tendency model.

In this paper, we propose an automatic Kansei fuzzy rule creating system using thesaurus. In Kansei Engineering, there are a lot of words for the expression of image. However, conventional methods cannot treat these words because it is too difficult to collect enough data of all words. The proposed system is an enhanced algorithm of the conventional method that the authors proposed. A set of fuzzy rules for only one impression word are extracted from the questionnaire responses by the conventional one. With the proposed system, sets of fuzzy rules for many words defined in the dictionary are extracted. The flow of the system consists of 3 steps: (1) calculation of the weights of thesaurus networks; (2) data collection; (3) extraction of fuzzy rules. Using a Japanese thesaurus dictionary, the sets of fuzzy rules for 1,195 impression words are extracted while the number of registered adjectives in the thesaurus is 2,822. From experimental results, the fuzzy rules extracted by the proposed system obtained higher accuracy than that extracted by the conventional one.

A personal Kansei modeling (PKM) system Is proposed in this paper. In Kansei modeling, tendency that is common to group members is usually discussed. However, treating personal tendency is becoming more and more important. With this system, a set of fuzzy rules are extracted through the analysis of Kansei data such as questionnaire responses. Generally, the amount of Kansei data taken from one person tends to be too small to analyze his/her Kansei. Basic idea of PKM system proposed in this paper is to create a common Kansei model from group data (first stage) before creating a personal Kansei model from personal data (second stage). In order to create a common Kansei model in the first stage, variance predictable general regression neural network (VP-GRNN), which is an enhanced version of GRNN, and fuzzy adaptive resonance theory (fuzzy ART) are employed in this system. A common model consists of a set of fuzzy rules, each associated with an adjustment factor, for the second stage. In the second stage, the fuzzy rules in the common model are adjusted using personal Kansei data to produce a set of fuzzy rules composing a personal Kansei model.

In this paper we present a new definition of neighborhood for particle swarm optimization (PSO) methods called area of influence. Area of influence (AOI) derives from the observation that in nature the effective exchange of information between individuals of a society deteriorates as their physical distance increases. In PSO with AOI, the loss of information exchange, ability with distance is simulated by making the exchange of information a function of the physical distance between particles in hyperspace. In this paper, we compare the AOI method to the standard PSO neighborhood methods, global best, local best, and von Neumann. We also introduce a local search method using reinitialization of velocity components based on the current search range. We show that AOI along with local search and a time-varying constriction coefficient provides strong benefits to several PSO algorithms. Results are presented using the standard benchmark functions from the PSO literature.

In this paper, we propose a new method for inverting feedforward neural networks. Inversion of neural networks means to find the inputs which produce given outputs. In general, this is an ill-posed problem whose solution isn't unique. Inversion using iterative optimization method (for example gradient descent, quasi-Newton method) is useful to this problem and it is called "iterative inversion". We propose a new iterative inversion using a Bottleneck Neural Network with Hidden layer's input units (BNNH), which we design on the basis of Bottleneck Neural Network (BNN). Compressing input space by BNNH, we reduce the dimension of search space, or input space to be searched with iterative inversion. With reduction of the search space's dimension, performance about computation time and accuracy is expected to become better. In experiments, the proposed method is applied to some examples. These results show the effectivity of the proposed method.

In this paper, we propose a machine composed of isomorphic modules creating support system using interactive evolutionary techniques. In manufacture of machines, design by modules is considered to be an important technique. The "machine" which is created using the proposed system consists of plural modules, and there are 12 kinds of modules, having different functions. The evolutionary techniques are applied to the individual structure where the modules are combined. By using isomorphous modules as composition elements of machines, we aim at acquisition of realizable and various machines. Moreover, by using interactive evolutionary techniques, the system can create a machine with suitable functions that a user desires, and a machine itself can stimulate a user's idea. The results of experiments show that various novel machines can be obtained.

This paper proposes a system for feeling estimation. It uses a simple electroencephalograph as an interface. If quantitative feeling presumption with information from a living body is made possible, it can be considered that a computer can appropriately deal with human's natural sensibility. Input data of proposed system is electroencephalogram (EEG) data of several feelings acquired using a simple electroencephalograph. Output of the system is one of some words associated with certain feelings states. One feature of proposed system is performing classification using the neural network. The neural networks have generalization ability and can deal with data including noise. Another feature is a low burden for the user by using a simple electroencephalograph. The proposed system is applicable to the quantitative evaluation replaced with a questionnaire. It is also applicable to handling the non-verbal information that is missing at communication.

In this paper, we propose a new neural network model termed semantic and episodic associative neural network (SEANN) for natural language processing. The SEANN can deal with both semantic memory and episodic memory by sentences represented in a form of a semantic network. In this model, both semantic memory and episodic memory are represented in triples-representation of concepts. Our model consists of concepts of sentences associative neural network (CSANN) and MAM using area representation. CSANN can recall sentences in a form of triples-representation, and MAM using area representation can recall plural triples-representations from a word. We have carried out computer experiments to confirm the validity of the SEANN for natural language processing. We have investigated that our model can recall plural semantic memories from one word, and can recall semantic memories concerning with episodic memory.

We propose a Japanese font automatic creating system reflecting user's Kansei. In the system, a user inputs the impression of a font to create. Since a user does not have to do direct design work a, user without special design skill can easily obtain various fonts reflecting his/her Kansei. The proposed system operates based on the genetic algorithm. The genetic algorithm consists of two parts: a font creation part and a font evaluation part. By repetition of creation and evaluation, the system outputs a font which meets the demand of a user. In addition, by repetition of the genetic algorithm, various fonts are obtained. Furthermore, the system has the learning ability to reflect each user's Kansei. Through the experiment, we have confirmed the availability of the system, such as the diversity of created fonts and the satisfaction of each user.

In this article, an idea divergent editor using analogy (IDEA) system has been proposed. The analogies between two concepts are learned using a neural network through training. First, the problem that a user wants to solve is input to the proposed IDEA system. Second, the IDEA system can draw effective analogies from a knowledge base containing large-scale and wide-ranging concepts for solutions to various problems. The user chooses one of the analogies. Then, the IDEA system presents questions that prompt the user to analyze the problem with reference to the background of the analogy. Answering the presented questions triggers off an inspiration of a solution. The IDEA system facilitates users' divergent thinking, which creates many fragments of ideas. We performed an experiment to examine the effectiveness of the IDEA system; the number of ideas generated is increased about 1.5∼2.0 times using the IDEA system. © 2003 Wiley Periodicals, Inc.

A new image segmentation method using a feature tree is proposed in this paper. The feature tree reflects the feature of an image. The proposed method is composed of two processes: (I) learning process and (II) clustering process. In the learning process, many efficient feature trees are made that construct an integrated tree. The integrated tree is used to segment images in the clustering process. Dividing an image is kept on from global point to local point. So, the proposed method can divide images considering not only the local property but also the global property. We applied the proposed method to some images, and obtained good results.

In this paper, we propose a system of evolving virtual robot structures in three-dimensional space. It is possible to obtain the structure of robots suitable for the virtual environment. We regard a robot as the combination of some functional parts (modules). To evolve the structure of robots, we use the technique of genetic programming. Grounds represented by some functions are assumed as the environment. Through computer experiments, it is confirmed that various structures of robots suitable to the environment can be obtained.

The authors propose a machine design support system using evolutionary techniques. The proposed system creates "machine" which consists of multiple parts. It is composed of two phases: the fundamental-parts-generation-phase, and the machine-generation-phase where the generated parts are combined. On each phase we use an evolutionary technique, genetic programming. The system generates parts itself, uses functional parts in advance, and expresses a shape's structure by tree structure. Therefore, it is possible to generate various and functional designs. We have implemented the system on PC, and carried out experiments to confirm that various novel objects can be obtained.

We propose a new system for a generation of a personal color-illustration from a facial image. The proposed system has the following features. It is possible to generate personal color-illustrations that take after the person in the input image. The system requires a small amount of user operation using its interactive interface to obtain his/her facial information. A skin region of the illustration is painted the same color as his/her skin. The illustration is deformed to have facial expression, and to exaggerate the facial features. Some accessory parts can be combined with the illustration. We confirmed the effectiveness of the proposed system by user survey.

In this paper, we propose a new system for recognition of scenery images, by learning. The proposed system consists of 2 parts. One is the part which infers based on absolute position of target. The other is the part which infers based on relative position of target. Each part consists of Fuzzy Inference Neural Network (FINN) which can extract fuzzy if-then rules automatically. So the system can extract knowledge about absolute position and relative position. Through computer experiments, it can be seen that the proposed system can recognize the image much correctly.

In this paper, we propose a knowledge processing system using improved chaotic associative memory (KPICAM). The proposed KPICAM is based on an improved chaotic associative memory (ICAM) composed of chaotic neurons. In the conventional chaotic neural network, when a stored pattern is given to the network as an external input continuously, around the input pattern is searched. The ICAM makes use of this property in order to separate superimposed patterns and to deal with many-to-many associations. In this research, the ICAM is applied to knowledge processing in which the knowledge is represented in a form of semantic network. The proposed KPICAM has the following features: (1) it can deal with the knowledge which is represented in a form of semantic network; (2) it can deal with characteristics inheritance; (3) it is robust for noisy input. A series of computer simulations shows the effectiveness of the proposed system.

The authors propose a chaotic associative memory for successive learning (CAMSL) using internal patterns. In the CAMSL, the learning process and the recall process are not divided. When an unstored pattern is given to the network, the CAMSL can learn the pattern successively. The CAMSL distinguishes an unstored pattern from the stored patterns. When a stored pattern is given, the CAMSL recalls the pattern. When an unstored pattern is given, the CAMSL changes the internal pattern for the input pattern by chaos and presents the other pattern candidates. When the CAMSL cannot recall the desired pattern, it learns the input pattern as an unstored pattern. We carried out a series of computer simulations and confirmed the effectiveness of the CAMSL.

We propose a self-growing learning vector quantization (SGLVQ). The proposed SGLVQ is constructed based on the self-organizing map (SOM) and the learning vector quantization (LVQ). Learning of the SGLVQ consists of 3 steps: SOM step, LVQ step, and rule extraction step. In the LVQ step, neurons are incremented and the size of the network is adjusted automatically. The incrementation of neurons enables additional learning and contributes to obtain high recognition ability. In the rule extraction step, rules can be extracted. Computer experiments show the improvement of the recognition rate, the ability of additional learning and extraction of the rules.

Proposes a visual nervous network for moving object recognition. This network is based on a visual neural net which has a hierarchical parallel structure. The network consists of three modules: the movement detection module, the moving object estimation module and the moving object recognition module. The movement detection module detects moving points and their direction. The moving object estimation module estimates the moving object based on motion features. The moving object recognition module recognizes the moving object based on its form feature. The network is able to recognize moving objects by uniting motion recognition and pattern recognition. We carried out computer simulations using real moving images to confirm the effectiveness of the proposed network.

In this paper, we propose a new hierarchical pulse neural network and its reinforcement learning algorithm with network extension. The proposed pulse neural network has three layers, and all of the neurons are pulse neurons. This network learns relations between input pulse sequences and the desired outputs by updating connection weights and by adding neurons dynamically. We carried out a computer simulation to confirm the performance of the proposed algorithm.

An effective image retrieval system is developed based on the use of neural networks (NNs). It takes advantages of association ability of multilayer NNs as matching engines which calculate similarities between a user's drawn sketch and the stored images. The NNs memorize pixel information of every size-reduced image (thumbnail) in the learning phase. In the retrieval phase, pixel information of a user's drawn rough sketch is inputted to the learned NNs and they estimate the candidates. Thus the system can retrieve candidates quickly and correctly by utilizing the parallelism and association ability of NNs. In addition, the system has learning capability: it can automatically extract features of a user's drawn sketch during the retrieval phase and can store them as additional information to improve the performance. The software for querying, including efficient graphical user interfaces, has been implemented and tested. The effectiveness of the proposed system has been investigated through various experimental tests.

We propose a new associative memory, named combined multi-winner associative memory (CMWAM), and study the bidirectional association properties. Unlike many conventional bidirectional associative memories, the proposed CMWAM can store and recall information hierarchically. The proposed CMWAM has the following features: 1) it can store analog information; 2) it can associate an episode from another episode; and 3) it is robust against noisy or partial information and it is robust against internal damage.

A new multidirectional associative memory (MAM), named combined multi-winner MAM (CMW-MAM) using distributed representation paradigm, is proposed and the association properties for storage of complex memory structure are studied. The conventional MAM stores and recalls information directly, however, the proposed CMW-MAM stores and recalls information hierarchically by the distributed representation which are formed automatically in the networks. The proposed CMW-MAM can handle analog information and can store and recall more complex memory structures. The robustness against incomplete information and internal damage is very strong.

In this paper, we propose an image segmentation by artificial life approach using autonomous agents. Each agent moves on the image according to some constant rules as follows; Each agent has features such as color. It moves onto a pixel which has the most similar features. It also puts pheromone on the pixels. The pheromone is based on the idea of chemical substances which have the property of keeping agents away. Each agent searches for the pixel which has smaller amount of pheromone and the most similar features. The locus of each agent becomes a segmented region. The proposed method has the following features: (1) it costs less amount of calculation; (2) it can be applied to a wide variety of images.

We propose a neural network model of visual system based on the feature integration theory. The proposed model has a structure based on the hierarchical structure of visual system and selectiveness of information by visual attention. The proposed model consists of two stages: the feature recognition stage and the feature integration stage. In the feature recognition stage, there are two modules: the form recognition module and the color recognition module. In these modules, information of form and color is separately processed in parallel. The form recognition module is constructed using the neocognitron, and the color recognition module is based on the LVQ neural network. The feature integration stage is based on the feature integration theory, which is a representative theory for explaining all phenomena occurring in visual system as a consistent process. We carried out computer simulations and confirmed that the proposed model can recognize plural objects and solve the binding problem.

We propose a novel neural network for four-term analogy based on area representation. It can deal with four-term analogy such as "teacher: student=doctor: ?". The proposed network is composed of three map layers and an input layer. The area representation method based on Kohonen feature map (KFM) is employed in order to represent knowledge, so that similar concepts are mapped in nearer area in the map layer. The proposed mechanism in the map layer can realize the movement of the excited area to the near area. We carried out some computer simulations and confirmed as follows: 1) similar concepts are mapped in the nearer area in the map layer; 2) the excited area moves among similar concepts; 3) the proposed network realizes four-term analogy; and 4) the network is robust for the lack of connections.

We propose a knowledge processing system using chaotic associative memory (KPCAM). The proposed KPCAM is based on a chaotic associative memory (CAM) composed of chaotic neurons. In the conventional chaotic neural network, when a stored pattern is given to the network as an external input continuously, the input pattern is searched. The CAM makes use of this property in order to separate the superimposed patterns and to deal with many-to-many associations. In this research, the CAM is applied to knowledge processing in which the knowledge is represented in a form of semantic network. The proposed KPCAM has the following features: 1) it can deal with the knowledge which is represented in a form of semantic network; 2) it can deal with characteristics inheritance; and 3) it is robust for noisy input. A series of computer simulations shows the effectiveness of the proposed system.

We propose a chaotic associative memory for sequential patterns (CAMSP). The proposed CAMSP is based on a chaotic associative memory composed of chaotic neurons. In the conventional chaotic neural network, when a stored pattern is given to the network as an external input continuously, the input pattern is searched. The CAM makes use of this property in order to separate the superimposed patterns. In this research, the CAM is applied to associations for sequential patterns. The proposed model has the following features: 1) it can deal with associations for the sequential patterns; 2) it can realize associations by considering patterns' history; and 3) it is robust for noisy input. A series of computer simulations shows the effectiveness of the proposed model.

In this paper, we propose a successive learning method in hetero-associative memories, such as Bidirectional Associative Memories and Multidirectional Associative Memories, using chaotic neural networks. It can distinguish unknown data from the stored known data and can learn the unknown data successively. The proposed model makes use of the difference in the response to the input data in order to distinguish unknown data from the stored known data. When input data is regarded as unknown data, it is memorized. Furthermore, the proposed model can estimate and learn correct data from noisy unknown data or incomplete unknown data by considering the temporal summation of the continuous data input. In addition, similarity to the physiological facts in the olfactory bulb of a rabbit found by Freeman are observed in the behavior of the proposed model. A series of computer simulations shows the effectiveness of the proposed model.

In this paper, first we derive a novel relaxation method for the system of linear inequalities and apply it to the learning for associative memories. Since the proposed intersection learning can guarantee the recall of all training data, it can greatly enlarge the storage capacity of associative memories. In addition, it requires much less weights renewal times than the conventional methods. We also propose a multimodule associative memory which can be learned by the intersection learning algorithm. The proposed associative memory can deal with many-to-many associations and it is applied to a knowledge processing task. Computer simulation results show the effectiveness of the proposed learning algorithm and associative memory.

We propose an episodic associative memory (EAM) called multi-winners self-organizing EAM (MWS-EAM) which can represent analog scenes of an episode distributedly and can recall the next scene from the distributed representation. The proposed MWS-EAM represents any scene of an episode into the representation layer distributedly, and then stores the relation of the scenes with its representation and the representation with next scene into the bottom up weights and the top down weights, respectively. The trained proposed MWS-EAM can recall all of the scenes in the stored episode by receiving only one scene.

We propose a new bidirectional associative memory (BAM) named multiwinners self-organizing bidirectional associative memory (MWS-BAM) in this paper. The proposed MWS-BAM has two processes for pattern pair storage: storage process and recall process. For the storage process, the proposed MWS-BAM can represent one piece of pattern pair distributedly in the distributed representation layer and stores the relation using the upward weights. In addition, the MWS-BAM can store the relation of the distributed representation and the other piece of pattern pair in the downward weights. For the recall process, the MWS-BAM can recall information bidirectionally: one piece of the stored pattern pair by receiving the other piece in the input-output layer for any stored pattern pairs.

A new system of knowledge extraction and recognition from scenery images is proposed in this paper. The system can extract different levels of knowledge automatically using Fuzzy Inference Neural Network (FINN). The proposed system consists of several Knowledge Extraction Networks (KENs). Each one is composed of FINN, and it can extract fuzzy if-then rules automatically. The KEN has an input-output (I/O) layer and two different-sized rule layers. The I/O layer includes the input part and the output part. The input part receives information on a pixel such as the position, the Intensity, the Hue and the Saturation. The output part receives the label of the corresponding pixel such as sky, mountains and woods, etc. The larger rule layer extracts detailed knowledge and it uses for the image recognition. On the other hand, the smaller rule layer extracts global knowledge which can correct contradiction of detailed knowledge and can remove trivial knowledge. It can be seen that the proposed system can recognize the image almost correctly by computer experiments. Knowledge is obtained by integrating rules from each KEN and then translating them into linguistic form. The extracted knowledge is quite natural.

We propose a music information retrieval system using complex-valued recurrent neural networks. Melodies can be treated as temporal sequences. The multilayer network using complex neurons with local feedback (MNCF) is used in the proposed system because of the high ability to deal with temporal sequences. The process of retrieval is as follows: first, the pitch and duration of each note are extracted from the key melody which is inputted from the MIDI keyboard. Second, they are inputted to some MNCFs. When the output values of each MNCF agree with the next input values for several times, it produces the database address. The user can listen to the candidate of the melody. In addition, the user can know the text information. The proposed system has the following other features: the user can input any parts of the melody he/she remembers as a key input; the human interface is excellent for users. The retrieval is performed in parallel because of the inherent parallelism of neural networks; the proposed system is robust for transposition and change of the tempo. We have evaluated the proposed system by experiment. The system retrieved about 91.5% of melodies correctly.

We propose a chaotic episodic associative memory (CEAM). It can deal with complex episodes which have common terms. Temporal associative memory (TAM) and episodic associative memory (EAM) have been proposed as models for episodic memory. However these models cannot deal with association of plural episodes that have common terms because the stored common patterns cause superimposed patterns. The proposed CEAM is based on the conventional TAM and has connections in the input layer for autoassociation. It also employs chaotic neurons in a part of the input layer. Each scene of the episodes is memorized together with its own contextual information. That is, the training set including common terms is converted into a form which doesn't include any common terms. The chaotic neurons in the input layer corresponding to contextual information change their states by chaos. As a result, the contextual information changes dynamically, which enables the CEAM to recall plural episodes that have common terms. A series of computer simulations shows the effectiveness of the proposed model.

Genetic programming (GP) is an algorithm of automatically inducing programs that solve or approximately solve problems. In creating programs the accuracy and the size are important factors. The paper introduces a modified genetic programming paradigm named GENPEI (modified genetic programming based on elastic artificial selection and improved minimum description length). In GENPEI, novel strategies for fitness evaluation and selection are used to create shorter programs and prevent premature convergence. GENPEI modifies the basic concepts of a simple genetic programming (SGP) to include EAS (elastic artificial selection) and IMDL (improved minimum description length). These methods are synergistically effective. In EAS, the best and the worst individuals are artificially selected and are stored for predetermined generations. After that they are reintroduced to the original pool. As for IMDL, it is devised to control the length of the programs. We performed computer simulations to confirm the effectiveness of the proposed GENPEI.

Artificial life (A-Life) is a new paradigm to realize a phenomena of life and to extract the hidden principles. One of the most attractive features in the A-Life approach is emergence: simple elements interact with each other based on lower level rules, and then the higher level complex phenomena can emerge by interaction. The paper proposes a method for knowledge extraction and integration based on an A-Life approach. The proposed system has two parts: the knowledge extraction network and the A-Life environment. The simple elements interact in the A-Life environment and the data is transferred to the knowledge extraction network. The knowledge is extracted in the form of rules in the rule layer and then they are fed back to the simple elements in the A-Life environment. We dealt with a path planning problem as an example of A-Life environment. In the simulation, we assumed a severe condition: the position of the goal was unknown to the robots. Since the robots did not know the goal in the initial condition, the trajectory by the first robot that reached the goal is very complicated. The trajectory data which the robots had taken were inputted to the knowledge extraction network to extract rules. The trajectories become smooth step by step because of the extracted rules. We extracted various kinds of the rules using several different simple environments. By using the rules extracted from the simpler environments, the robot could reach the goal in a more complex environment.

Although most of the conventional systems for object recognition have their own special targets, this paper gives a generic idea for a universal object recognition method. The proposed multi-module neural network (MMNN) is a hierarchical network with cascade connections, and consists of several modules which can detect specific features. MMNN is constructed based on the information processing of the visual nervous system such as a column structure in the Visual Area I and the hierarchical hypothesis of Hubel-Wiesel. As an example of a target object, we deal with human faces detection. This system consists of several modules in parallel which are trained to respond selectively to human face components: the eyes, the nose, and the mouth. Finally, the face area is detected by integrating the outputs of previous a cell layer. We carried out a lot of experiments using 100 images having complex background to conform the effectiveness of the proposed scheme. 83% of faces are detected correctly.

Proposes a coherent detector selection diversity system for fading channels. That is, in order to cope with fading channels, this system operates to switch between two phase-locked-loops (PLLs), one with the narrow loop bandwidth B/sub L/ and the other with the wide B/sub L/ by means of error-detection code. Moreover, the proposed system does not require differential encoding. Namely, it detects the signal with its absolute phase by means of error-detection. The effectiveness of this system compared with differential PSK is verified by computer simulation in fading channels; an indoor environment with maximum Doppler frequency of 5-15 Hz.

A digital signal processing (DSP) phase synchronizer using the method of least squares for fading channels (PS-FC) is presented. The proposed PS-FC has many characteristics which are quite different from conventional phase-locked loops (PLLs). For example, it can suppress noise using a modified method of least squares instead of a loop filter. The PS-FC can satisfy the phase tracking performance and the noise suppression performance simultaneously, which cannot be done by conventional PLLs. The pull-in range of the PS-FC is about three times as wide as that of the conventional PLL. It can estimate phase fluctuations due to fading much more accurately because of the modified method of least squares. Computer simulation shows its superior performance.>

A novel backpropagation algorithm with artificial selection is proposed. It is effective for both fast convergence and reduction of the number of hidden units. The main feature of the proposed algorithm is detection of the worst hidden unit. This is done by using the proposed badness factor, which indicates the badness of each hidden unit. It is the sum of backpropagated error components over all patterns for each hidden unit. For fast convergence, all the weights connected to the detected worst unit are reset to small random values at a suitable time. As for the reduction of hidden units, detected bad units are erased by precedent. Computer simulation results show the effectiveness of the proposed algorithm; for example, the number of hidden units in the EX-OR problems converge to two (theoretical number)

The conventional timing extraction method for block demodulation requires a large amount of calculation, a high-speed A/D (analog/digital) converter and a large memory size. A method that overcomes these disadvantages is proposed. It calculates accurate frequency and phase of symbol timing from only two extracted spectra by using a modified discrete Fourier transform (DFT). Thus it requires less calculation than the conventional method. In addition, interpolation of stored data is used to reduce the sampling rate. As a result the method can use a slower A/D converter and has a reduced memory requirement. The lower limit of the number of samples per symbol can be reduced to about 2.5.>

A digital signal processing (DSP)-type phase synchronizer based on the method of least squares (PS-MLS) is proposed that can replace the conventional circuits for phase synchronizers, such as phase-locked loops (PLLs). The construction of PS-MLS is different from that of PLLs. The proposed PS-MLS has advantageous characteristics compared with the conventional DSP-type PLLs and can satisfy three important points of PLLs simultaneously: fast acquisition, wide pull-in range, and good noise suppression. Only two steps are required for the acquisition (in a noise-less case), the condition to accomplish synchronization is relaxed, and is equal to the sampling theorem (fi>or=2fs), and the noise suppression performance is greatly improved compared with the conventional PLL.>

In this paper, we propose a Modulo-PCM with Multi-Quantizer (MPCM-MQ) which has high data compression efficiency because the proposed MPCM-MQ scheme can get rid of the anomaly errors by utilizing the multiquantizer method. By using this proposed scheme, we can improve its performance by about 10 [dB] ∼ 15 [dB] (1.5 [bits/sample] ∼ 1.8 [bits/sample]) in SQNR compared with the conventional MPCM-AQF scheme.

In this paper, we describe a new speech decoder for Modulo-PCM, which greatly reduces the "anomaly errors". Therefore, the proposed decoder can extend the dynamic range at the encoder input about 1.6-1.8 times compared to the conventional MPCM decoder.

This paper describes a new digital processing Phase Lock Loop (PLL) implemented on a high performance DSP, the DSSP1. Called the Linear Digital PLL (L-DPLL), the new PLL has linearty in its phase comparison characteristics. It gives fast acquision without increase in jitter, its pull-in range is wider; and its steady state errors and sampling frequency are lower than with conventional PLLs. It also does not require Automatic Gain Control.