In recent research, Tensor Product Representation (TPR) is applied for the systematic generalization task of deep neural networks by learning the compositional structure of data. However, such prior works show limited performance in discovering and representing the symbolic structure from unseen test data because their decomposition to the structural representations was incomplete. In this work, we propose an Attention-based Iterative Decomposition (AID) module designed to enhance the decomposition operations for the structured representations encoded from the sequential input data with TPR. Our AID can be easily adapted to any TPR-based model and provides enhanced systematic decomposition through a competitive attention mechanism between input features and structured representations. In our experiments, AID shows effectiveness by significantly improving the performance of TPR-based prior works on the series of systematic generalization tasks. Moreover, in the quantitative and qualitative evaluations, AID produces more compositional and well-bound structural representations than other works.

Neuro-symbolic neural networks have been extensively studied to integrate symbolic operations with neural networks, thereby improving systematic generalization. Specifically, Tensor Product Representation (TPR) framework enables neural networks to perform differentiable symbolic operations by encoding the symbolic structure of data within vector spaces. However, TPR-based neural networks often struggle to decompose unseen data into structured TPR representations, undermining their symbolic operations. To address this decomposition problem, we propose a Discrete Dictionary-based Decomposition (D3) layer designed to enhance the decomposition capabilities of TPR-based models. D3 employs discrete, learnable key-value dictionaries trained to capture symbolic features essential for decomposition operations. It leverages the prior knowledge acquired during training to generate structured TPR representations by mapping input data to pre-learned symbolic features within these dictionaries. D3 is a straightforward drop-in layer that can be seamlessly integrated into any TPR-based model without modifications. Our experimental results demonstrate that D3 significantly improves the systematic generalization of various TPR-based models while requiring fewer additional parameters. Notably, D3 outperforms baseline models on the synthetic task that demands the systematic decomposition of unseen combinatorial data.

Intracranial hemorrhage (ICH) is a type of stroke with a high mortality rate and failing to localize even minor ICH can put a patient's life at risk. However, its patterns are diverse in shapes and sizes and, sometimes, even hard to recognize its existence. Therefore, it is challenging to accurately detect and localize diverse ICH patterns. In this article, we propose a novel Perihematomal Edema Guided Scale Adaptive R-CNN (PESA R-CNN) for accurate segmentation of various size hemorrhages with the goal of minimizing missed hemorrhage regions. In our approach, we design a Center Surround Difference U-Net (CSD U-Net) to incorporate Perihematomal Edema (PHE) for more accurate Region of Interest (RoI) generation. We trained CSD U-Net to predict PHE and hemorrhage regions as targets in a weakly supervised manner and utilized its prediction results to generate RoI. By including more informative features of PHE around hemorrhage, this enhanced RoI generation allows a model to reduce the false-negative rate. Furthermore, these expanded RoIs are aligned with the Scale Adaptive RoI Align (SARA) module based on their size to prevent the loss of fine-scale information and small hemorrhage patterns. Each scale adaptively aligned RoI is processed with the corresponding separate segmentation network of Multi-Scale Segmentation Network (MSSN), which integrates the results from each scale's segmentation network. In experiments, our model shows significant improvement on dice coefficient (0.697) and Hausdorff distance (12.918), compared to all other segmentation models. It also minimizes the number of missing small hemorrhage regions and enhances overall segmentation performance on diverse ICH patterns.

Multi-hop question generation (QG) is the process of generating answer related questions, which requires aggregating multiple pieces of information and reasoning from different parts of the texts. This is opposed to single-hop QG which generates questions from sentences containing an answer in a given paragraph. Single-hop QG requires no reasoning or complexity, while multi-hop QG often requires logical reasoning to derive an answer related question, making it a dual task. Not enough research has been made on the multi-hop QG due to its complexity. Also, a question should be created using the question type and words related to the correct answer as a prompt so that multi-hop questions can get more information. In this view, we propose a new type-dependent prompt cycleQAG (cyclic question-answer-generation), with a cycle consistency loss in which QG and Question Answering (QA) are learnt in a cyclic manner. The novelty is that the cycle consistency loss uses the negative cross entropy to generate syntactically diverse questions that enable selecting different word representations. Empirical evaluation on the multi-hop dataset with automatic and human evaluation metrics outperforms the baseline model by about 10.38% based on ROUGE score.

The aim of this study is to recognize human emotions from electroencephalographic (EEG) signals using deep neural networks. Large training data is an important prerequisite for successful implementation of deep neural networks. In this view, we propose an independent component analysis (ICA) - evolution based data augmentation method. This method performs ICA to extract and accumulate clean independent components (ICs) of each class. The new ICs are generated by selection which uses a fitness function such as mutual information (MI) and crossover in component space. Data augmentation is done by performing mutation, and crossover on generated data in sensor space. Since EEG signals are non-stationary, with time-varying frequency contents, emotional patterns associated with EEG are detected in the time-frequency (TF) domain using a spectrogram. To extract emotion related features from a spectrogram, we train an ensemble convolutional neural networks (CNNs) with convolutional kernels in time and frequency axes. The information integrated over both the axes is concatenated and fed to long short-term memory (LSTM). We used the benchmark DEAP dataset for emotion classification to evaluate our approach. The results highlight the potential of proposed ICA-evolution based data augmentation and an ensemble CNNs with LSTM model for emotion recognition.

Modern digital personal assistants interact with users through voice.Therefore, they heavily rely on automatic speech recognition (ASR) in order to convert speech to text and perform further tasks.We introduce EBERT, which stands for EmbraceBERT, with the goal of extracting more robust latent representations for the task of noisy ASR text classification.Conventionally, BERT is fine-tuned for downstream classification tasks using only the [CLS] starter token, with the remaining tokens being discarded.We propose using all encoded transformer tokens and further encode them using a novel attentive embracement layer and multi-head attention layer.This approach uses the otherwise discarded tokens as a source of additional information and the multihead attention in conjunction with the attentive embracement layer to select important features from clean data during training.This allows for the extraction of a robust latent vector resulting in improved classification performance during testing when presented with noisy inputs.We show the impact of our model on both the Chatbot and Snips corpora for intent classification with ASR error.Results, in terms of F1-score and mean between 10 runs, show that our model significantly outperforms the baseline model.

Mel-frequency filter bank (MFB) based approaches have the advantage of higher learning speeds compared to using the raw spectrum due to a smaller number of features. However, speech generators with the MFB approach require an additional computationally expensive vocoder for the training process. The pre- and post-processing needed by the MFB and the vocoder is not essential to convert human voices, because it is possible to use only the raw spectrum to generate different style of voices with clear pronunciation. In this paper, we introduce a vocoder-free end-to-end voice conversion method using a transformer network to alleviate the computational burden from additional pre- and post-processing. Our transformer-based architecture, which does not have any CNN or RNN layers, has shown the benefit of learning fast while solving the limitation of sequential computation of the conventional RNN. For this reason, our model is a fast and effective approach to convert realistic voices using raw spectra in a parallel manner to generate different style of voices with clear pronunciation. Furthermore, we can get an adapted MFB for speech recognition by multiplying the converted magnitude with the phase information, and therefore our conversion model is also suitable for speaker adaptation. We perform our voice conversion experiments on TIDIGITS-dataset using the naturalness, similarity, and clarity with Mean Opinion Score as metrics1.

Atrial fibrillation (AF) is the most prevalent cardiac arrhythmia. The atrial beat is irregular during AF, which causes blood flow hardly. This may cause blood clot formation and cardioembolic strokes. Computer-aided devices may assist cardiologists in diagnosing heart rhythm disorders better. From this viewpoint, we attempt to identify the premature atrial complexes (PACs) to predict the occurrence of AF by using electrocardiogram (ECG) spectrograms. Convolutional neural networks (CNN) models such as ResNet and Wide-ResNet are used to predict the prelude of AF. Regularization constraints are used to deal with the imbalanced and small number of samples in the minority premature AF class. Sensitivity regularization investigates small variations in premature AF samples. It highlights more representative features that distinguish the PACs from the normal rhythm. On the other hand, orthogonality regularization removes the interference between negatively correlated feature weights. It places constraints on capturing similar patterns with slight differences. This constraint allows convergence to a better feature representation with fewer weight redundancies. We propose a combination of sensitivity and orthogonality penalty terms to the cost function of ResNet to decrease the overfitting and obtain a superior representation. The re-sampling class distribution method is also utilized to mitigate the issue of imbalanced data. The proposed method shows better AF prediction for highly imbalanced data with a small number of samples.

Most of the conventional Seq2seq based chit-chat models can analyze and process one or two sentences at a time and are trained to answer specific patterns. However, in a real chit-chat conversation, one sentence is interpreted in various ways according to the preceding context. It is often difficult to make a different response for the same input with the existing Seq2seq based chit-chat models. This makes it difficult for Seq2seq models to understand the previous context in a multi-turn chit-chat conversation. To overcome this problem, the dialogue generation models should have an external memory to store the contextual information related to a conversation. In this paper, we propose a new dialogue generation model, which uses differentiable neural computer (DNC) in the conventional Seq2seq model named Seq-DNC-seq. The proposed Seq-DNC-seq model incorporates external memory into the conventional Seq2seq structure to generate an appropriate dialogue based on the memory of previous conversations. Experimental results show that the proposed Seq-DNC-seq model successfully generates multi-turn chit-chat and the output sentences differ depending on the previous sentence even with the same input text. This not only helps the agent to overcome the existing limitations of chit-chat conversation but also understand the context in longer conversation.

In humans, perception and action (PA) possess cyclically causal relations. In this paper, we propose a new PA-based cyclic learning framework to autonomously enhance the depth-estimation accuracy of a humanoid robot and perform given behavioral tasks. The proposed method integrates the concepts of sensory invariance-driven action and object-size invariance to autonomously enhance the depth-estimation accuracy. If the depth estimation is reliable, the reinforcement learning framework is used to generate goal-directed actions of a humanoid robot based on a perceived environment. Iterative PA cycles of a robot autonomously refine its depth-estimation. The proposed method is evaluated using a humanoid robot (NAO) with stereo cameras, and the experimental results demonstrate that the proposed framework is effective for autonomously enhancing both the depth-estimation accuracy and the action-generation performance.

This work tackles sentence correction in the lan-guage domain by approaching it as a sequence to sequence (seq2seq) problem with the help of temporal hierarchies. It does so by implementing a Multiple Timescales model of the Gated Recurrent Unit (MTGRU) in a Recurrent Neural Network (RNN) Encoder-Decoder framework, which can perform more meaningful data abstraction even in the presence of errors. The proposed language correction model is compared to three baseline models: conventional RNN, Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU); by using a newly built dataset that consists of incorrect and correct sentences as input and target respectively. The result shows that the MTGRU model has a better generalization performance and outperforms all three models on the BLEU-n evaluation metric.

Representing multiple compositions of human language has been a difficult task due to the complex hierarchical and compositional nature of language. Hierarchical structures are one of the architectures which can be used to capture such compositionalities. In this paper, we introduce temporal hierarchies to the Neural Language Model (NLM) with the help of a Deep Gated Recurrent Neural Network with adaptive timescales to help represent multiple compositions of language. We demonstrate that by representing multiple compositions of language in a deep recurrent neural network architecture, we can improve the performance of Language Models without complex hierarchical architectures. We report the performance of the proposed model using the popular Penn Treebank (PTB) dataset. The results show that by using the multiple timescale concept in an NLM, we can achieve better perplexities compared to the existing baselines.

An image retrieval system is a technique for browsing, searching and retrieving images from a big database of digital images. In this paper, we propose a new content-based image retrieval system that can solve the object and scene recognition problems and categorize similar images. The proposed model consists of a deep structure support vector machine with Gaussian mixture model, which is combined with human-like top-down selective attention model using growing fuzzy topology adaptive resonant theory (GFTART) network and scene understanding using GIST. The results suggest that the proposed model has better performance than other recent methods used in this field.

In this paper, we propose a framework that can prevent accidents due to careless or inattentive driving by providing the necessary traffic information to the driver. The proposed system complements the driver by providing the missed cognitive information regarding the traffic. The proposed system is divided into three parts. First, the system checks the condition of the driver in real time, and detects the status of the driver in terms of driving ability. Second, we propose bottom-up and top-down processes based on Traffic Information Saliency Map (TISM) which contains the distribution corresponding to the external road information using bottom-up traffic information saliency map and top-down importance information such as pedestrian and traffic light detection results. Computer experimental results show that the proposed method works well for monitoring of internal situation for driver's attention as well as external environment.

Palmprint recognition is a challenging problem, mainly due to low quality of the patterns, variation in focal lens distance, large nonlinear deformations caused by contactless image acquisition system, and computational complexity for the large image size of typical palmprints. This paper proposes a new contactless biometric system using features of palm texture extracted from the single hand image acquired from a digital camera. In this work, we propose to apply convolutional neural network (CNN) for palmprint recognition. The results demonstrate that the extracted local and general features using CNN are invariant to image rotation, translation, and scale variations.

We propose a smart cane with a face recognition system to help the blind in recognizing human faces. This system detects and recognizes faces around them. The result of the detection is informed to the blind person through a vibration pattern. The proposed system was designed to be used in real-time and is equipped with a camera mounted on the glasses, a vibration motor attached to the cane and a mobile computer. The camera attached to the glasses sends image to mobile computer. The mobile computer extracts features from the image and then detects the face using Adaboost. We use the modified census transform (MCT) descriptor for feature extraction. After face detection, the information regarding the detected face image is gathered. We used compressed sensing with L2-norm as a classifier. Cane is equipped with a Bluetooth module and receives a person's information from the mobile computer. The cane generates vibration patterns unique to each person as to inform a blind person about the identity of the detected person using the camera. Hence, the blind people can know the person standing in front of them.

Recently, there has been tremendous development in education contents for foreign language learning. Based on these trends, IT has provided educational contents development using e-learning and broadcast media. But conventional educational contents are non-interactive presents an impediment to provide user's specific service. To develop a user friendly language education tool, we propose an intelligent learning tool based on user's eye movement and brain waves. By analyzing these features, the proposed system detects if the given word is known or unknown to the user while learning a foreign language. Then it searches its meaning and provides a vocabulary list of unknown words to users in real time. The proposed model provides a tool which enables self-directed learning. We assume that the proposed system can improve users' learning achievements and satisfaction.

This paper proposes an agent to assist human cognition in memorizing multiple human faces by analyzing user's eye gaze points. The gaze point which is the direction of sight is obtained by the infrared camera on a glass-type agent with the help of an embedded module. The gaze information is then combined with the image captured by the frontal camera to identify the location of the face that the user is looking at among several faces. The gaze detection and face selection with tracking are performed in embedded modules attached to the glass-type agent, and the recognition of the selected facial images is performed and shown on a mobile computer connected via wireless network. The major contribution of the proposed work is the use of eye gaze direction to select faces of interest, and provide information regarding the faces to improve human memory capability in recalling the faces.

In Advanced Driving Assistance Systems (ADASs), monitoring the driver's cognitive status during driving is considered as an important issue. Because, most of the accidents in the automotive sector occur due to the driver's misinterpretation or lack of sufficient information regarding the situation. In order to prevent these accidents, current ADASs include lane departure warning systems, vehicle detection systems, advanced cruise control systems, etc. In a particular driving scenario, the amount of information available to the driver regarding a situation can be judged by monitoring the driver's gaze (internal information) and distributions corresponding to the forward traffic (external information). Therefore, to provide sufficient information to the driver regarding a driving scenario it is essential to integrate the internal and external information which is lacking in the current ADASs. In this paper, we use 3D pose estimate algorithm (POSIT) to estimate driver's attention area. In order to estimate the distributions corresponding to the forward traffic we employ Bottom-up Saliency map. To integrate the internal and external information we use conditional mutual information.

Recognition of human intention is an important issue in human-robot interaction research and allows a robot to respond adequately according to human's wish. In this paper, we discuss how robots can infer human intention by learning affordance, a concept used to represent the relation between an agent and its environment. Learning of the robot, to understand human and its interaction with environment, is achieved within the framework of action-perception cycle. The action-perception cycle explains how an intelligent agent learns and enhances its ability continuously by interacting with its surrounding. The proposed intention recognition and recommendation system includes several key functions such as joint attention, object recognition, affordance model, motion understanding module and so on. The experimental results show high successful recognition performance and the plausibility of the proposed system.

Picture is worth a thousand words. With changing life styles and technology advancement, visual or pictorial communication is preferred. We present a system to generate a pictogram for simple Korean sentences. The final pictogram integrates information about the object (about which something is said), the background (the environment) and the emotion of the user. The proposed system is divided into two parts. First is the registration part, which saves personal information and face image of the user. The second part searches corresponding images for words, downloads them and finally integrates all of them together to along with user's emotion to generate a single pictogram.

Monitoring the concentration level of a learner is important to maximize the learning effect, giving proper feedback on tasks and to understand the performance of learners in tasks. In this paper, we propose a personal concentration level monitoring system when a user performs an online task on a computer by analyzing his/her pupillary response and eye-blinking pattern. We use low-priced web camera to detect eye blinking pattern and a portable eye tracker to detect pupillary response. Experimental results show good performance of the proposed concentration level monitoring system and suggest that it can be used for various real applications such as intelligent tutoring system, e-learning system, etc.

This paper presents a real-time hand gesture detection and recognition method. Proposed method consists of three steps - detection, validation and recognition. In the detection stage, several areas, estimated to contain hand shapes are detected by random forest hand detector over the whole image. The next steps are validation and recognition stages. In order to check whether each area contains hand or not, we used Linear Discriminant Analysis. The proposed work is based on the assumption that samples with similar posture are distributed near each other in high dimensional space. So, training data used for random forest are also analyzed in three dimensional space. In the reduced dimensional space, we can determine decision conditions for validation and classification. After detecting exact area of hand, we need to search for hand just in the nearby area. It reduces processing time for hand detection process.

We present our ongoing work on a creativity assistance tool called I-get. The tool is based on the hypothesis that perceptual similarity between a pair of images, at a subconscious level, plays a key role in generating creative conceptual associations and metaphorical interpretations. The tool "I-get" is designed to assist users to create novel ideas and metaphorical associations primed by algorithmic perceptual similarity between two images and alternative conceptual associations given by users.

Images can convey emotion just like music. If that's so, then it might be possible that, given an image, one can obtain a music that can produce a similar reaction from the listener/viewer. The challenge lies in how to do that. In this paper, we analyze the image using the HSV color space model and assume that each one of the three components have a relation with basic music elements, like tone, pitch, rhythm and loudness. The image is then scanned from left to right and top to bottom in order to generate a sequence of notes. In the end, the emotional Mean Opinion Score (MOS) is used to evaluate the performance of the proposed method. This work could prove to be a very important contribution to the field of HCI because it can improve the interaction between computers and humans who are visually and/or hearing impaired. In the current work, we only consider two emotions; positive and negative.

Motion understanding and regeneration are two basic aspects of human-agent interaction. One important function of agents is to represent human's activities. For better interaction with human, robot agents should not only do something following human's order, but also be able to understand or even play some actions. Multiple Timescale Recurrent Neural Networks (MTRNN) is believed to be an efficient tool for robots action generation. In our previous work, we extended the concept of MTRNN and developed Supervised MTRNN for motion recognition. In this paper, we use Conditional Restricted Boltzmann Machine (CRBM) to initialize Supervised MTRNN and accelerate the training speed of Supervised MTRNN. Experiment results show that our method can greatly increase the training speed without losing much performance.

Deep learning methods allow a classifier to learn features automatically through multiple layers of training. In a deep learning process, low-level features are abstracted into high-level features. In this paper, we propose a new probabilistic deep learning method that combines a discriminative model, namely, Support Vector Machine (SVM), with a generative model, namely, Gaussian Mixture Model (GMM). Combining the SVM with the GMM, we can represent a new input feature for deeper layer training of uncertain data in current layer construction. Bayesian rule is used to re-represent the output data of the previous layer of the SVM with GMM to serve as the input data for the next deep layer. As a result, deep features are reliably extracted without additional feature extraction efforts, using multiple layers of the SVM with GMM. Experimental results show that the proposed deep structure model allows for an easier classification of the uncertain data through multiple-layer training and it gives more accurate results.

Intention understanding is a basic requirement for human-machine interaction. Action classification and object affordance recognition are two possible ways to understand human intention. In this study, Multiple Timescale Recurrent Neural Network (MTRNN) is adapted to analyze human action. Supervised MTRNN, which is an extension of Continuous Timescale Recurrent Neural Network (CTRNN), is used for action and intention classification. On the other hand, deep learning algorithms proved to be efficient in understanding complex concepts in complex real world environment. Stacked denoising auto-encoder (SDA) is used to extract human implicit intention related information from the observed objects. A feature based object detection method namely Speeded Up Robust Features (SURF) is also used to find the object information. Object affordance describes the interactions between agent and the environment. In this paper, we propose an intention recognition system using `action classification' and `object affordance information'. Experimental result shows that supervised MTRNN is able to use different information in different time period and improve the intention recognition rate by cooperating with the SDA.

Human cognition is the result of an interaction of several complex cognitive processes with limited capabilities. Therefore, the primary objective of human cognitive augmentation is to assist and expand these limited human cognitive capabilities independently or together. In this study, we propose a glass-type human augmented cognition system, which attempts to actively assist human memory functions by providing relevant, necessary and intended information by constantly assessing intention of the user. To achieve this, we exploit selective attention and intention processes. Although the system can be used in various real-life scenarios, we test the performance of the system in a person identity scenario. To detect the intended face, the system analyses the gaze points and change in pupil size to determine the intention of the user. An assessment of the gaze points and change in pupil size together indicates that the user intends to know the identity and information about the person in question. Then, the system retrieves several clues through speech recognition system and retrieves relevant information about the face, which is finally displayed through head-mounted display. We present the performance of several components of the system. Our results show that the active and relevant assistance based on users' intention significantly helps the enhancement of memory functions.

Recent developments in vision systems such as distributed smart cameras have encouraged researchers to develop advanced computer vision applications suitable to embedded platforms. In the embedded surveillance system, where memory and computing resources are limited, simple and efficient computer vision algorithms are required. In this letter, we present a moving object detection method for real-time traffic surveillance applications. The proposed method is a combination of a genetic dynamic saliency map (GDSM), which is an improved version of dynamic saliency map (DSM) and background subtraction. The experimental results show the effectiveness of the proposed method in detecting moving objects.

Differential Evolution (DE), a global optimization algorithm based on the concepts of Darwinian evolution, is popular for its simplicity and effectiveness in solving numerous real-world optimization problems in real-valued spaces. The effectiveness of DE is due to the differential mutation operator that allows DE to automatically adjust between the exploration/exploitation in its search moves. However, the performance of DE is dependent on the setting of control parameters such as the mutation factor and the crossover probability. Therefore, to obtain optimal performance preliminary tuning of the numerical parameters, which is quite timing consuming, is needed. Recently, different parameter adaptation techniques, which can automatically update the control parameters to appropriate values to suit the characteristics of optimization problems, have been proposed. However, most of the adaptation techniques try to adapt each of the parameter individually but do not take into account interaction between the parameters that are being adapted. In this paper, we introduce a DE self-adaptive scheme that takes into account the parameters dependencies by means of a multivariate probabilistic technique based on Gaussian Adaptation working on the parameter space. The performance of the DE algorithm with the proposed parameter adaptation scheme is evaluated on the benchmark problems designed for CEC 2014.

We propose a novel approach to recognize positive or negative emotions from acoustic signals in movies by extracting musical components such as tempo, loudness and melody and then by applying ANFIS Model with fuzzy clustering. In order to extract emotional features in acoustic signals, we first transform the sound into a spectrogram. The spectrogram visually represents characteristic information of sound such as tempo, loudness and melody. Then, we apply the fuzzy model on spectrogram to get the effective emotion features of sound. The extracted tempo, loudness and melody information is used as inputs for an adaptive neuro-fuzzy inference system (ANFIS) with fuzzy c-means clustering (FCM). Finally, the ANFIS classifies the sound as positive or negative emotion, which is compared with a mean opinion score of human in test movies.

Most of the machine learning algorithms particularly suffer from the plasticity-stability dilemma. In this paper, we propose a model that adopts two types of memories i.e. short-term memory (STM) and long-term memory (LTM), which share their information through control processes called rehearsal and recall to alleviate the dilemma. In addition, the proposed model tries to integrate the advantages of generative and discriminative classifiers by employing them in STM and LTM respectively. Experimental results show the importance of rehearsal and recall process in improving the performance of the algorithm.

In this paper, we present a method to track moving vehicles in low frame rate videos which are common in embedded traffic surveillance systems. In general, an embedded surveillance system has limited memory and computing resources, and thus the frame rate of video dramatically decreases. Hence, the features of moving vehicles such as shapes and sizes vary dramatically which is difficult to be handled using appearance and/or feature based conventional methods. In the proposed model, the probability distribution of a tracking vehicle in the next frame is predicted based on a hypothesis which is constructed by trajectory identification model using manifold learning. By the projecting on the low dimensional manifold, the probabilistic similarity between the observed and the predicted probability distributions of the tracking vehicles is measured. The probabilistic distribution with maximum similarity among several candidate hypotheses in the trajectory identification models is considered to include spatial information to track a moving vehicle. Experimental results show the effectiveness of the proposed method in tracking moving vehicles, even when the shapes, positions and sizes change rapidly.

Differential Evolution (DE) is a simple and effective approach for solving numerical optimization problems. However, the performance of DE is sensitive to the choice of the mutation and crossover strategies and their associated control parameters. Therefore, to obtain optimal performance, time consuming parameter tuning is necessary. In DE, different mutation and crossover strategies with different parameter settings can be appropriate during different stages of the evolution. Therefore, to obtain optimal performance using DE, various adaptation and self-adaptation techniques have been proposed. Recently, a DE algorithm with an ensemble of parameters and strategies (EPSDE) was proposed. In EPSDE, a pool of distinct mutation and crossover strategies along with a pool of values for each control parameter coexists throughout the evolution process and competes to produce offspring. The performance of EPSDE degrades if the population members get struck with a combination of strategies and parameters values that produce successful offspring but lead to premature convergence in the due course of the evolution. In this paper, we try to improve the performance of the EPSDE algorithm with the help of a surrogate model that assists in generating competitive trial vectors corresponding to each parent in every generation of the evolution. The proposed algorithm is referred to as surrogate model assisted EPSDE (SMA-EPSDE) and employs a simple Kriging model to construct the surrogate. The performance of EPSDE is evaluated on a set of 17 bound-constrained problems and is compared with state-of-the-art algorithms.

Principal Component Analysis (PCA) is one of the most widely used subspace projection technique for face recognition. In subspace methods like PCA, feature selection is fundamental to obtain better face recognition. However, the problem of finding a subset of features from a high dimensional feature set is NP-hard. Therefore, to solve the feature selection problem, heuristic methods such as evolutionary algorithms are gaining importance. In many face recognition applications, due to the small sample size (SSS) problem, it is difficult to construct a single strong classifier. Recently, ensemble learning in face recognition is gaining significance due to its ability to overcome the SSS problem. In this paper, the NP-hard problem of finding the best subset of the extracted PCA features for face recognition is solved by using the differential evolution (DE) algorithm and is referred to as FS-DE. The feature subset is obtained by maximizing the class separation in the training data. We also present an ensemble based approach for face recognition (En-FR), where different subsets of PCA features are obtained by maximizing the distance between a subset of classes of the training data instead of whole classes. The subsets of the classes are obtained by bagging and overlap each other. Each subset of the PCA features selected is used for face recognition and all the outputs are combined by a simple majority voting. The proposed algorithms, FS-DE and En-FR, are evaluated on four wellknown face databases and the performance is compared with the PCA and Fisher's LDA algorithms.

Interpretation of human implicit intention is crucial in the development of an efficient nonverbal human computer interaction system. According to cognitive visuo-motor theory, the human eye movements and pupillary responses are rich source of information about the human intention and behavior. It has been observed that under conditions of constant illumination and accommodation, pupil size varies systematically in relation to a variety of physiological and psychological factors, such as level of mental effort. It is well known that pupillary responses could be used to measure the differences in cognitive load under various tasks. In this paper, we try to detect the transition between the different human implicit intents based on the pupil state analysis. In real-world environment, the pupillary response can be influenced by various external factors like intensity and size of the image. To overcome the influence of the external factors, we develop a robust baseline model. The proposed approach detects the transition of the human's implicit intent from navigational intent to informational intent and vice versa during a visual stimulus. The approach also detects the transition among the different states of the informational intent such as informational intent generation, informational intent maintenance and informational intent disappear.

The aim of cognitive augmentation is to expand the intrinsically limited human's cognitive abilities caused by cognitive impairment or disability. In order to assist the human's limited cognitive ability, we are trying to develop a human augmented cognition system that aims to provide the appropriate information actively corresponding to what user intents to do. In this paper, we mainly address the probabilistic human intention modeling for cognitive augmentation, and its overall process. The types of implicit intention such as navigational and informational intention can be predicted by using fixation count and length induced by eyeball movement. Also, the gradient of pupil size variation is used to detect the transition point between navigational intent and the informational intent. A Naïve Bayes classifier is used as a tool for the extraction of query keywords to search and retrieve specific information from personalized knowledge database according to the successive series of attended objects according to a specific informational intent in a situation. The experimental results show that the probabilistic human intention model is suitable for achieving the ultimate purpose of the cognitive augmentation.

In this paper, we propose a new incremental two-directional two-dimensional principal component analysis (I(2D) PCA) to efficiently recognize human faces. For implementing a real time face recognition system in an embedded system, the reduction of computational load as well as memory of a feature extraction algorithm is very important issue. The (2D) PCA is faster than the conventional PCA. From memory capacity point of view, the incremental PCA is very efficient algorithm by adapting the eigensapce only using a new incoming sample data without memorizing all of previous trained data. In order to construct an efficient algorithm with less memory and small computational load, we propose a new feature extraction method by combining the IPCA and the (2D)2PCA. To evaluate the performance of the proposed I(2D)2PCA, a series of experiments were performed on two face image databases: ORL and Yale face databases. The experimental results show that the proposed feature extraction method is efficient by reducing the memory while computational load is nearly similar to (2D)2PCA.

Most accurate gaze trackers commonly use near IR (infrared ray) illuminators to detect a pupil rather than an iris because the pupil detection provides higher accuracy for implementing a gaze tracker and it is easier to detect the pupil under IR illumination. However, the active IR illuminating methods directly emit energies to human eyes and also generate heats to an embedded mobile device. Thus, it may be uncomfortable and unstable to utilize an active IR illuminating method in an embedded mobile device as a gaze tracker for a long time. In this paper, we propose a new gaze tracking method using a common USB camera, in which a multilayer perceptron is applied to estimate the pupil's location using iris area information localized in a face area detected from a captured image. The pupil location information as teaching target signals for the neural network is obtained from off-line experiments using an IR camera with an illuminator. And localized iris area information obtained from on-line experiments using a common USB camera is used as input signals of the neural network. Experimental results show that the proposed method plausibly performs the pupil estimation by the multilayer perceptron and successfully generates gaze tracking by an additional calibration process.

In this paper, we propose an incremental 2-directional 2-dimensional linear discriminant analysis (I-(2D)2LDA) for multitask pattern recognition (MTPR) problems in which a chunk of training data for a particular task are given sequentially and the task is switched to another related task one after another. In I-(2D)2LDA, a discriminant space of the current task spanned by 2 types of discriminant vectors is augmented with effective discriminant vectors that are selected from other tasks based on the class separability. We call the selective augmentation of discriminant vectors knowledge transfer of feature space. In the experiments, the proposed I-(2D)2LDA is evaluated for the three tasks using the ORL face data set: person identification (Task 1), gender recognition (Task 2), and young-senior discrimination (Task 3). The results show that the knowledge transfer works well for Tasks 2 and 3; that is, the test performance of gender recognition and that of young-senior discrimination are enhanced.

Humans can efficiently perceive arbitrary visual objects based on incremental learning mechanism and selective attention function. In this paper, we propose a new top-down attention model based on human visual attention mechanism, which considers both relative feature based bottom-up saliency and goal oriented top-down attention. The proposed model can generate top-down bias signals of form and color features for a specific object, which draw attention to find a desired object by an incremental learning mechanism together with object feature representation scheme. A growing fuzzy topology adaptive resonance theory (GFTART) model is proposed by adapting a growing cell structure (GCS) unit into a conventional fuzzy ART, by which the proliferation problem of the conventional fuzzy ART can be enhanced. The proposed GFTART plays two important roles for object color and form biased attention; one is to incrementally learn and memorize color and form features of arbitrary objects, and the other is to generate top-down bias signal for selectively attending to a target object. Experimental results show that the proposed model performs well in successfully focusing on given target objects, as well as incrementally perceiving arbitrary objects in natural scenes.

In this paper we propose a novel “fuzzy-GIST” for analyzing the subject specific emotion reflected by a natural scene, considering both the human emotional state and the visual features extracted from the scene image. According to the relationship between emotional factors and the characters of image, we incorporate the fuzzy concept to extract emotional features using L*C*H* color and orientation information. On the other hand, after various pre-processing of emotional electroencephalography (EEG), we treat emotional relevant EEG features using the fuzzy logic based on possibility theory rather than widely used conventional probability theory to generate the semantic feature of the human emotions. Fuzzy-GIST consists of both semantic visual information and linguistic EEG feature, it is used to represent emotional gist of a natural scene in a semantic level. We use a neuro-fuzzy inference model to infer the emotion evoked by an image, and the feedback from the subject is used for supervising the learning as well as evaluating the performance of the proposed scheme. The experiment results show the possibility to recognize four different emotions for a given dataset. Moreover, the analysis of human emotional status and visual information also can serve for the study of interaction between human subject and machine, and possibly the development of new brain machine interface.

In this paper, a new feature extraction algorithm considering both two-directional two-dimensional principal component analysis ((2D)2PCA) and independent component analysis (ICA), called (2D)2PCA-ICA, is proposed for face representation. This algorithm analyzes the principal components of image vectors on 2D matrices by simultaneously considering the row and column directions as opposed to the standard PCA based on 1D vectors, and transforming those principal components to the independent components that maximize the non-Gaussianity of the sources. These two major techniques such as (2D)2PCA and ICA are used sequentially in order to obtain the most efficient features that properly describe a whole set of human faces in face databases. The proposed algorithm is applied to the face recognition problem. Simulation results on ORL and Yale B face databases shows that the proposed algorithm achieves high average success rate in face recognition compared with other models.

We propose an autonomous robot vision system that is applied to develop an intelligent artificial officemate. In order to operate the proposed system in real environment, it is very important for the officemate to be able to adapt to an environmental changes that may occur in an indoor environment. Novelty detection is one of essential functions for the officemate to detect a situation change. The proposed system can indicate a novel scene and a scene change based on a visual selective attention module. Moreover, it can adaptively acquire new information based on incremental object perception, face recognition, and emotion representation. In order to implement an on-line officemate system, we implement an efficient model by simplification and optimization procedure which can reduce the computation load. Experimental results show that the developed system successfully identifies a change of natural scenes and incrementally learns an arbitral object and a face, and it can also extend its knowledge through interaction with human supervisor.

We propose an autonomous blind spot monitoring method using a morphology-based saliency map (SM) model and the method of combining scale invariant feature transform (SIFT) with mean-shift tracking algorithm. The proposed method decides a region of interest (ROI) which includes the blind spot from the successive image frames obtained by side-view cameras. Topology information of the salient areas obtained from the SM model is used to detect a candidate of dangerous situations in the ROI, and the SIFT algorithm is considered for verifying whether the localized candidate area contains an automobile. We developed a modified mean-shift algorithm to track the detected automobile in a blind spot area. The modified mean-shift algorithm uses the orientation probability histogram for tracking the automobile around the localized area. Experimental results show that the proposed algorithm successfully provides an alarm signal to the driver in a dangerous situations caused by approaching an automobile at side-view.

Artificial emotion study will be of utmost importance in future artificial intelligence research. In this paper, an emotion understanding system based on brain activity and ldquoGISTrdquo is newly proposed to categorize emotions reflected by natural scenes. According to the strong relationship of human emotion and the brain activity, functional magnetic resonance imaging (fMRI) and electroencephalography (EEG) are used to analyze and classify emotional states stimulated by a natural scene. The ldquoGISTrdquo is used to represent the emotional gist of the natural scene. In other words, by taking the way human brain responding to the same stimulus into consideration, a machine will be able to visually extract the emotional features of natural scenes and achieve interaction with a human in terms of emotional sharing. The experimental results show that positive and negative emotions can be distinguished, and a monkey robot head that can share emotion with human subject during watching an image is implemented.

Recognition regarding the changing environment is an essential role for survival. Novelty scene detection plays an important role in evoking self motivation to adapt to changing environments and efficiently to bring about new knowledge. In this paper, we propose a biologically motivated novelty scene detection model, which is implemented by a proposed incremental computation model. Every input scene is represented by visual scan path topology and the energy signatures, which are obtained from a saliency map generated by a low level top-down visual attention model in conjunction with a bottom-up saliency map model. The obtained representation for an input scene is used as the input for the incremental computation model in order to memorize scenes and detect novelty scenes. The computer experimental results show that the proposed model successfully indicates a novelty for natural color input scenes in a natural visual environment.

In this paper, we propose an object selective attention and perception system, which was implemented by integrating a specific object preferable attention model with an incremental object perception model. The object oriented attention model can selectively pay attention to the candidates of an object in natural scenes based on a bottom-up selective attention model in conjunction with a top-down biased attention mechanism for a specific object. A generative model based on an incremental Bayesian parameter estimation is considered in order to perceive arbitrary objects in the attended areas. Combining an object oriented attention model with general object perception model, the developed system cannot only pay attention to a specific target object but can also memorize the characteristics of task nonspecific objects in an incremental manner. Experimental results show that the developed system generates good performance in successfully focusing on the target objects as well as incrementally perceiving objects in natural scenes.

The brain computer interface (BCI) is a computer interface system which is based upon brain activity. In order to develop an efficient BCI system, we need to consider suitable settings of the mental tasks as well as the optimal location of the EEG sensors. We used an fMRI experiment in an attempt to determine those optimal settings. According to our experiment, there might be some relationships between fMRI experiment data and EEG data even though they have different physical properties. Based on the fMRI experiment with various kinds of mental task, we could find a possibility to enhance BCI system performance with small number of EEG sensors.

We propose a new method for a region of interest (ROI) based image segmentation that uses biologically motivated selective attention model. One of the most important issues in image segmentation based on a region of interest (ROI) is how to decide upon a semantic object region according to a specific purpose. The proposed saliency map model in conjunction with a top-down Fuzzy adaptive resonance theory (ART) model for human interaction can generate a scan path that contains a plausible area in a natural scene. In order to extract an interesting region generated by the saliency map model, we propose a new region of interest (ROI) extraction algorithm using scale salient information and multiple features such as a intensity, edge, R+G-, and B+Y-color to reflect more exact salient regions. Computer experimental results show that the proposed model can successfully segment an ROI boundary in natural scenes and computer graphics.

In this paper, we propose a biologically motivated face preference selective attention system to identify a face within complex natural scenes. In order to localize a face in natural scenes, we have developed a task-specific selective attention model which integrates the conventional bottom-up saliency map with punishment and rewarding functions, with top-down attention and bias signals, according to a given task. The color-filtered intensity, color opponent, and edge of the winner color opponent features are intensified for the biasing of skin color in order to identify a face. Computer experimental results have shown that the proposed model successfully identifies multiple faces within a complex environment. In addition, we have implemented a robot vision system which will be used for an autonomous mental development system.

We propose a biologically motivated novelty detection model of a scene that can give a robust performance for natural color scenes with an affine transformed field of view, as well as noisy scenes in a dynamic visual environment. Novelty detection is an essential property for developmental robots. A topology of a visual scan path of an input scene and an energy signature for the corresponding visual scan path are obtained and considered when deciding on a novelty occurrence in an input scene. The visual scan path is generated by a low-level top-down attention model in conjunction with a bottom-up saliency map model.

We propose a new biologically motivated model to localize or detect faces in natural color input scene. The proposed model integrates a bottom-up saliency mechanism for extracting features from an input image and a top-down perceptual mechanism for detecting faces using the results of the bottom-up processing. For bottom-up feature extraction, we consider the roles of cells in our visual receptor for edge detection and cone opponency, and also reflect the roles of the lateral geniculate nucleus to find a symmetrical property of an interesting object such as shape and pattern. Also, independent component analysis (ICA) is used to find a filter that can generate a salient region from feature maps constructed by edge, color opponency and symmetry information, which models the role of redundancy reduction in the primary visual cortex. For the top down perceptional processing to detect faces, we partially model the role of the inferior temporal areas, which plays an important role for face recognition. Computer experimental results show that the proposed model successfully indicates faces in natural scenes.

This paper presents a robust control scheme using a multilayer neural network with the error backpropagation learning algorithm. The multilayer neural network acts as a compensator of the conventional sliding mode controller to improve the control performance when initial assumptions of uncertainty bounds of system parameters are not valid. The proposed controller is applied to control a robot manipulator operating under the sea which has large uncertainties such as the buoyancy, the drag force, wave effects, currents, and the added mass/moment of inertia. Computer simulation results show that the proposed control scheme gives an effective path way to cope with those unexpected large uncertainties.

We propose a new computational model for mimicking the behavior of a human eye movement during saccades. The different characteristics of two types of saccades, such as a reflexive saccade and an intentional saccade, are reflected on the proposed model. We divided the visual pathway for generating a saccadic eye movement into three parts, of which each part was modeled using different neural networks. The visual pathway from the visual receptors to the visual cortex including the frontal eye field was modeled by the self-organizing feature map, and the visual pathway from the visual cortex to the superior colliculus was modeled by a modified learning vector quantization network. The visual pathway front the superior colliculus to the motoneuron is modeled by a multilayer neural network with backpropagation learning algorithm. Experimental results from computer simulation show that the proposed computational model is able to mimic well the behavior of the human eye movement for two different saccades.