In this paper, we propose an approach to improve detection of event related potential (ERP) component using hidden process model, which enables estimating the trial-to-trial variability of ERP latency to overcome limitation of the conventional averaging method for extracting ERP components. By using HPM, which is a generative model for estimating underlying process that has unknown onset timing, we can estimate responses of assumed processes underlying cognitive functions and the probability distribution of the onset timing of each process. We applied HPM to ERP data obtained during the oddball task and distinguished ERPs induced by target or nontarget stimuli. We designed 2-process and 3-process HPMs and estimated the responses of each process in these HPMs. Then, we compared these responses with ERP waveforms obtained by conventional averaging. As a result, the waveforms of the estimated response from each model resembled that of averaged ERP while the peak amplitude was higher in estimated responses than in averaged ERP. In addition, the difference of the area under curve between target and nontarget condition was clearer in estimated responses than in averaged ERP. This suggests that HPM might be able to overcome the latency variability of ERP components to estimate more exact components, which will enhance differentiating ERP components between conditions in an ERP study.

The tactile properties of objects are important for robotic dexterous manipulation. An increasing number of attempts have recently been made to enable tactile information processing in robotic hand via tactile sensors. However, it remains relatively unexplored how to build tactile information processing models. In this study, we aimed to develop a spiking neural network (SNN) based on neural information processing mechanisms in sensory afferents. The SNN processes electrical signals collected from tactile sensor arrays attached to the gripper of the robotic hand while grasping objects with different shapes. We converted each of 42 -channel sensor signals from 2 arrays of 21 sensors into a spike train using the Izhikevich model, which was then fed to the SNN. The synaptic weights of the SNN were learned by the Hebbian learning through pair-based spike timing- dependent plasticity (STDP) algorithm. In addition, we implemented lateral inhibition of the second-layer neurons based on unsupervised learning similar to the one used in self-organizing maps, resulting in a winner-takes-all network. By this unsupervised learning, SNN could learn to discriminate the shape of objects via tactile sensing. In particular, it demonstrated object shape recognition with 100% accuracy. The proposed model could be useful for robots manipulating objects with tactile senses.

The present neuromarketing study aims to predict how consumers evaluate brand extension from their brain activity. Brand extension refers to the use of well-established brand name to new goods or service. In our experiment, participants evaluated whether a given brand extension sample was acceptable during functional magnetic resonance imaging (fMRI) scanning. Brand extension samples included parent beverage brand name and extended goods name from beverage or household appliance categories. We pre-processed 3-D fMRI image data and extracted 2-D feature images based on a preliminary study. To overcome the limited number of fMRI samples, we conducted image augmentation for the training dataset. A deep neural network model with the convolutional neural network (CNN) architecture was used to classify fMRI images in response to visual presentation of each brand extension sample into one of the two classes: acceptable vs. non-acceptable. The train classifier was tested using 10-fold cross validation. Our model estimated the evaluation of brand extension with high accuracy (90.4%). This result indicates that one can predict how a consumer evaluates a new brand extension proposal only by their brain activity.

This study investigates brain activity related to drivers' anxiety induced by external situations during watching driving video using electroencephalography (EEG). We conducted a spectral analysis of 4 frequency bands, including theta, alpha, beta, and gamma bands, of frontal brain oscillations known to be involved in emotion recognition. Furthermore, we calculated frontal asymmetry and beta/alpha ratio with respect to the valence-arousal domain. We confirmed that driver's perspective video could induce anxiety in the viewers and found three frontal EEG features pronounced in anxious situations: 1) smaller alpha power reflecting more cognitive control; 2) right dominance of theta and beta oscillations; and 3) greater difference between beta and alpha. These EEG features may provide a basis for developing technology to monitor drivers' emotional states in smart cars.

In this study, we investigated a feasibility to predict a daily mental stress level from heart rate variability (HRV) using a photoplethysmography (PPG) sensor in the wristband-type wearable device. We performed an experiment in which each participant measured their PPG signals for 30 s using the wristband three times a day for a week. The recorded signals were transmitted to and stored at a smartphone via the Bluetooth link by custom-made software. At the end of each day, participants also self-evaluated their mental stress level using the perceived stress scale (PSS). A preprocessing procedure was used to remove environmental artifacts in the PPG signal and HRV was estimated from the PPG signal by the detection of PPG peaks. We then extracted a low-frequency (0.04Hz - 0.15Hz) / high-frequency (0.15Hz - 0.4Hz) feature of HRV using the autoregressive (AR) model. A linear regression model predicted the self-reported mental stress level from the HRV features. Prediction accuracy was 86.35% on average across the participants. The proposed method could demonstrate a feasibility of developing a mobile health solution that predicts a personal mental stress level using HRV measured by a wristband PPG sensor.

In this work, we studied the feasibility of MSP (Multispectral Skin Photomatrix) module as a new wearable biometrie devices. The MSP device measures optical patterns of the wrist skin tissue. Optical patterns consist of 2 χ 16 photocurrent intensities of photodiode arrays, which is generated by optical transmission and diffuse reflection of photons from LED light sources with variable wavelengths into the wrist skin tissue. Optical patterns detected by the MSP device provide information on both surface and subsurface characteristics of the human skin tissue. Totally 73 subjects have been participated in this study, they showed their unique characteristics, as determined by several wavelengths of light. The experimental results show that the best personal identification accuracy can be acquired using a combination of infrared light and yellow light. This novel biometric device, the MSP module, exhibited an excellent false acceptance rate (FAR) of 0.292% and a false rejection rate (FRR) of 3.539%, which are similar value of another biometric technologies such as voice or face recognition, etc. From these experimental results, we found that people exhibit unique optical patterns of their inner-wrist skin tissue and this uniqueness could be used for developing novel high-accuracy personal identification devices.

Little has been known about the process of learning in a human population during continuous goal-directed tasks. Most models of human-like decision-making based multi-agent systems have been focused on learning by discrete feedback. Also, the multi-agent systems in engineering seldom involve the cognitive concept. In this study, we design a multi-agent system based task in which a group of persons controls a common actuator (i.e. a computer cursor) to achieve the task goal (i.e. target acquisition). The cursor movement provides human agents real-time visual feedback. Each subject is given the control capability of a specific cursor direction that is not informed a priori. The population vector, which has been developed to relate a population of motor neurons to arm direction, is used to drive the cursor movement. Learning occurs in the group of human agents as each agent becomes aware of their contributions (i.e. their controlling directions) to achieving the task goal, leveraging only continuous visual feedback of cursor position on the screen. Moreover, the system level learning is exhibited even though some agents failed to learn their true directions with high accuracy. The experimental results show that the multiple human agents can learn to gain the control of a cursor based on visual feedbacks. In addition, we examine how the learning occurs in the individual level and the collective level.

Brain-computer interfaces rely on accurate decoding of cortical activity to understand intended action. Algorithms for neural decoding can be broadly categorized into two groups: direct versus generative methods. Two generative models, the population vector algorithm (PVA) and the Kalman filter (KF), have been widely used for many intracortical BCI studies, where KF generally showed superior decoding to PVA. However, little has been known for which conditions each algorithm works properly and how KF translates the ensemble information. To address these questions, we performed a simulation study and demonstrated that KF and PVA worked congruently for uniformly distributed preferred directions (PDs) whereas KF outperformed PVA for non-uniform PDs. In addition, we showed that KF decoded better than PVA for low signal-to-noise ratio (SNR) or a small ensemble size. The results suggest that KF may decode direction better than PVA with non-uniform PDs or with low SNR and small ensemble size.

Neural motor prostheses (NMPs) require the accurate decoding of motor cortical population activity for the control of an artificial motor system. Previous work on cortical decoding for NMPs has focused on the recovery of hand kinematics. Human NMPs however may require the control of computer cursors or robotic devices with very different physical and dynamical properties. Here we show that the firing rates of cells in the primary motor cortex of non-human primates can be used to control the parameters of an artificial physical system exhibiting realistic dynamics. The model represents 2D hand motion in terms of a point mass connected to a system of idealized springs. The nonlinear spring coefficients are estimated from the firing rates of neurons in the motor cortex. We evaluate linear and a nonlinear decoding algorithms using neural recordings from two monkeys performing two different tasks. We found that the decoded spring coefficients produced accurate hand trajectories compared with state-of-the-art methods for direct decoding of hand kinematics. Furthermore, using a physically-based system produced decoded movements that were more “natural” in that their frequency spectrum more closely matched that of natural hand movements.

Brain machine interfaces (BMI) utilize linear or non-linear models to map the neural activity to the associated behavior which is typically the 2D or 3D hand position of a primate. Linear models are plagued by the massive disparity of the input and output dimensions thereby leading to poor generalization. A solution would be to use non-linear models like the recurrent multi-layer perceptron (RMLP) that provide parsimonious mapping functions with better generalization. However, this results in a drastic increase in the training complexity, which can be critical for practical use of a BMI. This paper bridges the gap between superior performance per trained weight and model learning complexity. Towards this end, we propose to use echo state networks (ESN) to transform the neuronal firing activity into a higher dimensional space and then derive an optimal sparse linear mapping in the transformed space to match the hand position. The sparse mapping is obtained using a weight constrained cost function whose optimal solution is determined using a stochastic gradient algorithm.

The challenge for functional magnetic resonance imaging (fMRI) is to determine when and where the response due to an external stimulus occurs in the images. The temporal clustering analysis (TCA) method has been used to study brain activity after eating and drinking, in both the time and spatial domains. We propose a new method, competitive subspace projection (CSP), to represent data optimally compared to other subspace projection methods. This method is used to detect spatial and temporal activation patterns in fMRI associated with such behavior. The CSP and TCA methods are compared using both synthetic and real fMRI data. The results on both data sets show consistent conclusions can be drawn from these two methods while CSP is observed to have a better noise rejection capability than TCA.

In this paper, we show that the sparse channel estimation problem can be formulated as a regularization problem between mean squared error (MSE) and the L/sub 1/-norm constraint of the channel impulse response. A simple adaptive method to solve regularization problem using the convolution inequality for entropy is proposed. Performance of this proposed regularization method is compared to the Wiener filter, the matching pursuit (IMP) algorithm and the information criterion based method. The results show that the estimate of the sparse channel using the MSE criterion with the L/sub 1/-norm constraint outperforms the Wiener filter and the conventional sparse solution methods in terms of MSE of the estimates and the generalization performance.

Linear or non-linear models are used in brain machine interfaces (BIMIs) to map the neural activity to the associated behavior, typically the primate's hand position. Linear models assume a linear relationship between neural activity and hand position that may not be the case. A solution would be time-delay neural network (TDNN) that provides effectively a nonlinear combination of linear models. However, this model results in a drastic increase of free parameters and slow convergence when trained by an error backpropagation learning rule. We propose to train the TDNN by scaled conjugate gradient, which avoids time-consuming linear search, coupled with weight decay to reduce the free parameters number and produce generally faster convergence.

Principal components analysis (PCA), being the most optimal linear mapper in a least-squares (LS) sense, has been predominantly used in subspace-based signal processing methods. In system identification problems, optimal subspace projections must span the joint space of the input and output of the unknown system. In this scenario, subspaces determined by the principal components of the input or the desired signal alone do not embed key information, which lies in the joint space. We first propose a hybrid subspace projection method that finds optimal projections in the joint space. The concepts behind this method are firmly rooted in statistical theory. We then derive adaptive learning algorithms to estimate the subspace projections. Finally, we show the superiority of the new framework in solving system identification problems in noisy environments.

Recent research has demonstrated that linear model are able to estimate hand positions using populations of action potentials collected in the pre-motor and motor cortical areas of a primate's brain. One of the applications of this result is to restore movement in patients suffering from paralysis. To implement this technology in real-time, reliable and accurate signal processing models that produce sufficient small error in the estimated hand positions are required. In this paper, we propose the hybrid model approach that combines competitive linear filters with a neural network. The mapping performance of our approach is compared with a single Wiener filter during reaching movements. Our approach demonstrates more accurate estimations.

We are working on mapping multi-channel neural spike data, recorded from multiple cortical areas of an owl monkey, to corresponding 3D monkey arm positions. In earlier work on this mapping task, we observed that continuous function approximators (such as artificial neural networks) have difficulty in jointly estimating 3D arm positions for two distinct cases-namely, when the monkey's arm is stationary and when it is moving. Therefore, we propose a multiple-model approach that first classifies neural spike data into two classes, corresponding to two states of the monkey's arm: (1) stationary and (2) moving. Then, the output of this classifier is used as a gating mechanism for subsequent continuous models, with one model per class. In this paper, we first motivate and discuss our approach. Next, we present encouraging results for the classifier stage, based on hidden Markov models (HMMs), and also for the entire bimodal mapping system. Finally, we conclude with a discussion of the results and suggest future avenues of research.