Diffusion models have significantly facilitated the customization of input video with target appearance while maintaining its motion patterns. To distill the motion information from video frames, existing works often estimate motion representations as frame difference or correlation in pixel-/feature-space. Despite its simplicity, these methods have unexplored limitations, including lack of understanding of global motion context, and the introduction of motion-independent spatial distortions. To address this, we present Spectral Motion Alignment (SMA), a novel framework that refines and aligns motion representations in the spectral domain. Specifically, SMA learns spectral motion representations, facilitating the learning of whole-frame global motion dynamics, and effectively mitigating motion-independent artifacts. Extensive experiments demonstrate SMA's efficacy in improving motion transfer while maintaining computational efficiency and compatibility across various video customization frameworks.

Diffusion distillation models effectively accelerate reverse sampling by compressing the process into fewer steps. However, these models still exhibit a performance gap compared to their pre-trained diffusion model counterparts, exacerbated by distribution shifts and accumulated errors during multi-step sampling. To address this, we introduce Distillation++, a novel inference-time distillation framework that reduces this gap by incorporating teacher-guided refinement during sampling. Inspired by recent advances in conditional sampling, our approach recasts student model sampling as a proximal optimization problem with a score distillation sampling loss (SDS). To this end, we integrate distillation optimization during reverse sampling, which can be viewed as teacher guidance that drives student sampling trajectory towards the clean manifold using pre-trained diffusion models. Thus, Distillation++ improves the denoising process in real-time without additional source data or fine-tuning. Distillation++ demonstrates substantial improvements over state-of-the-art distillation baselines, particularly in early sampling stages, positioning itself as a robust guided sampling process crafted for diffusion distillation models. Code: https://github.com/geonyeong-park/inference_distillation.

Spurious correlations can cause strong biases in deep neural networks, impairing generalization ability. While most existing debiasing methods require full supervision on either spurious attributes or target labels, training a debiased model from a limited amount of both annotations is still an open question. To address this issue, we investigate an interesting phenomenon using the spectral analysis of latent representations: spuriously correlated attributes make neural networks inductively biased towards encoding lower effective rank representations. We also show that a rank regularization can amplify this bias in a way that encourages highly correlated features. Leveraging these findings, we propose a self-supervised debiasing framework potentially compatible with unlabeled samples. Specifically, we first pretrain a biased encoder in a self-supervised manner with the rank regularization, serving as a semantic bottleneck to enforce the encoder to learn the spuriously correlated attributes. This biased encoder is then used to discover and upweight bias-conflicting samples in a downstream task, serving as a boosting to effectively debias the main model. Remarkably, the proposed debiasing framework significantly improves the generalization performance of self-supervised learning baselines and, in some cases, even outperforms state-of-the-art supervised debiasing approaches.

The brain prepares for learning even before interacting with the environment, by refining and optimizing its structures through spontaneous neural activity that resembles random noise. However, the mechanism of such a process has yet to be understood, and it is unclear whether this process can benefit the algorithm of machine learning. Here, we study this issue using a neural network with a feedback alignment algorithm, demonstrating that pretraining neural networks with random noise increases the learning efficiency as well as generalization abilities without weight transport. First, we found that random noise training modifies forward weights to match backward synaptic feedback, which is necessary for teaching errors by feedback alignment. As a result, a network with pre-aligned weights learns notably faster and reaches higher accuracy than a network without random noise training, even comparable to the backpropagation algorithm. We also found that the effective dimensionality of weights decreases in a network pretrained with random noise. This pre-regularization allows the network to learn simple solutions of a low rank, reducing the generalization error during subsequent training. This also enables the network to robustly generalize a novel, out-of-distribution dataset. Lastly, we confirmed that random noise pretraining reduces the amount of meta-loss, enhancing the network ability to adapt to various tasks. Overall, our results suggest that random noise training with feedback alignment offers a straightforward yet effective method of pretraining that facilitates quick and reliable learning without weight transport.

Neural networks are often biased to spuriously correlated features that provide misleading statistical evidence that does not generalize. This raises an interesting question: “Does an optimal unbiased functional subnetwork exist in a severely biased network? If so, how to extract such subnetwork?” While empirical evidence has been accumulated about the existence of such unbiased subnetworks, these observations are mainly based on the guidance of ground-truth unbiased samples. Thus, it is unexplored how to discover the optimal subnetworks with biased training datasets in practice. To address this, here we first present our theoretical insight that alerts potential limitations of existing algorithms in exploring unbiased subnetworks in the presence of strong spurious correlations. We then further elucidate the importance of bias-conflicting samples on structure learning. Motivated by these observations, we propose a Debiased Contrastive Weight Pruning (DCWP) algorithm, which probes unbiased subnetworks without expensive group annotations. Experimental results demonstrate that our approach significantly outperforms state-of-the-art debiasing methods despite its considerable reduction in the number of parameters.

Despite the remarkable performance of text-to-image diffusion models in image generation tasks, recent studies have raised the issue that generated images sometimes cannot capture the intended semantic contents of the text prompts, which phenomenon is often called semantic misalignment. To address this, here we present a novel energy-based model (EBM) framework for adaptive context control by modeling the posterior of context vectors. Specifically, we first formulate EBMs of latent image representations and text embeddings in each cross-attention layer of the denoising autoencoder. Then, we obtain the gradient of the log posterior of context vectors, which can be updated and transferred to the subsequent cross-attention layer, thereby implicitly minimizing a nested hierarchy of energy functions. Our latent EBMs further allow zero-shot compositional generation as a linear combination of cross-attention outputs from different contexts. Using extensive experiments, we demonstrate that the proposed method is highly effective in handling various image generation tasks, including multi-concept generation, text-guided image inpainting, and real and synthetic image editing. Code: https://github.com/EnergyAttention/Energy-Based-CrossAttention.

Human skeleton-based action recognition offers a valuable means to understand the intricacies of human behavior because it can handle the complex relationships between physical constraints and intention. Although several studies have focused on encoding a skeleton, less attention has been paid to embed this information into the latent representations of human action. InfoGCN proposes a learning framework for action recognition combining a novel learning objective and an encoding method. First, we design an information bottleneck-based learning objective to guide the model to learn informative but compact latent representations. To provide discriminative information for classifying action, we introduce attention-based graph convolution that captures the context-dependent intrinsic topology of human action. In addition, we present a multi-modal representation of the skeleton using the relative position of joints, designed to provide complementary spatial information for joints. InfoGcn11Code is available at github.com/stnoahl/infogcn surpasses the known state-of-the-art on multiple skeleton-based action recognition benchmarks with the accuracy of 93.0% on NTU RGB+D 60 cross-subject split, 89.8% on NTU RGB+D 120 cross-subject split, and 97.0% on NW-UCLA.

Social learning, copying other's behavior without actual experience, offers a cost-effective means of knowledge acquisition. However, it raises the fundamental question of which individuals have reliable information: successful individuals versus the majority. The former and the latter are known respectively as success-based and conformist social learning strategies. We show here that while the success-based strategy fully exploits the benign environment of low uncertainly, it fails in uncertain environments. On the other hand, the conformist strategy can effectively mitigate this adverse effect. Based on these findings, we hypothesized that meta-control of individual and social learning strategies provides effective and sample-efficient learning in volatile and uncertain environments. Simulations on a set of environments with various levels of volatility and uncertainty confirmed our hypothesis. The results imply that meta-control of social learning affords agents the leverage to resolve environmental uncertainty with minimal exploration cost, by exploiting others' learning as an external knowledge base.

The estimation and inference of human predictive uncertainty have great potential to improve the sampling efficiency and prediction reliability of human-in-the-loop systems for smart healthcare, smart education, and human-computer interactions. Predictive uncertainty in humans is highly interpretable, but its measurement is poorly accessible. Contrarily, the predictive uncertainty of machine learning models, albeit with poor interpretability, is relatively easily accessible. Here, we demonstrate that the poor accessibility of human uncertainty can be resolved by exploiting simple and universally accessible deterministic neural networks. We propose a new model for human uncertainty inference, called proxy ensemble network (PEN). Simulations with a few benchmark datasets demonstrated that the model can efficiently learn human uncertainty from a small amount of data. To show its applicability in real-world problems, we performed behavioral experiments, in which 64 physicians classified medical images and reported their level of confidence. We showed that the PEN could predict both the uncertainty range and diagnoses given by subjects with high accuracy. Our results demonstrate the ability of machine learning in guiding human decision making; it can also help humans in learning more efficiently and accurately. To the best of our knowledge, this is the first study that explored the possibility of accessing human uncertainty via the lens of deterministic neural networks.

While multi-step adversarial training is widely popular as an effective defense method against strong adversarial attacks, its computational cost is notoriously expensive, compared to standard training. Several single-step adversarial training methods have been proposed to mitigate the above-mentioned overhead cost; however, their performance is not sufficiently reliable depending on the optimization setting. To overcome such limitations, we deviate from the existing input-space-based adversarial training regime and propose a single-step latent adversarial training method (SLAT), which leverages the gradients of latent representation as the latent adversarial perturbation. We demonstrate that the ℓ1norm of feature gradients is implicitly regularized through the adopted latent perturbation, thereby recovering local linearity and ensuring reliable performance, compared to the existing single-step adversarial training methods. Because latent perturbation is based on the gradients of the latent representations which can be obtained for free in the process of input gradients computation, the proposed method costs roughly the same time as the fast gradient sign method. Experiment results demonstrate that the proposed method, despite its structural simplicity, outperforms state-of-the-art accelerated adversarial training methods.

Adversarial learning strategy has demonstrated remarkable performance in dealing with single-source Domain Adaptation (DA) problems, and it has recently been applied to Multi-source DA (MDA) problems. Although most existing MDA strategies rely on a multiple domain discriminator setting, its effect on the latent space representations has been poorly understood. Here we adopt an information-theoretic approach to identify and. resolve the potential adverse effect of the multiple domain discriminators on MDA: disintegration of domain-discriminative information, limited computational scalability, and a large variance in the gradient of the loss during training. We examine the above issues by situating adversarial DA in the context of information regularization. This also provides a theoretical justification for using a single and unified domain discriminator. Based on this idea, we implement a novel neural architecture called a Multi-source Information-regularized Adaptation Networks (MIAN). Large-scale experiments demonstrate that MIAN, despite its structural simplicity, reliably and significantly outperforms other state-of-the-art methods.

Depression is characterized by deficits in the reinforcement learning (RL) process. Although many computational and neural studies have extended our knowledge of the impact of depression on RL, most focus on habitual control (model-free RL), yielding a relatively poor understanding of goal-directed control (model-based RL) and arbitration control to find a balance between the two. We investigated the effects of subclinical depression on model-based and model-free learning in the prefrontal-striatal circuitry. First, we found that subclinical depression is associated with the attenuated state and reward prediction error representation in the insula and caudate. Critically, we found that it accompanies the disrupted arbitration control between model-based and model-free learning in the predominantly inferior lateral prefrontal cortex and frontopolar cortex. We also found that depression undermines the ability to exploit viable options, called exploitation sensitivity. These findings characterize how subclinical depression influences different levels of the decision-making hierarchy, advancing previous conflicting views that depression simply influences either habitual or goal-directed control. Our study creates possibilities for various clinical applications, such as early diagnosis and behavioral therapy design.

Controllability perception significantly influences motivated behavior and emotion and requires an estimation of one's influence on an environment. Previous studies have shown that an agent can infer controllability by observing contingency between one's own action and outcome if there are no other outcome-relevant agents in an environment. However, if there are multiple agents who can influence the outcome, estimation of one's genuine controllability requires exclusion of other agents' possible influence. Here, we first investigated a computational and neural mechanism of controllability inference in a multi-agent setting. Our novel multi-agent Bayesian controllability inference model showed that other people's action-outcome contingency information is integrated with one's own action-outcome contingency to infer controllability, which can be explained as a Bayesian inference. Model-based functional MRI analyses showed that multi-agent Bayesian controllability inference recruits the temporoparietal junction (TPJ) and striatum. Then, this inferred controllability information was leveraged to increase motivated behavior in the vmPFC. These results generalize the previously known role of the striatum and vmPFC in single-agent controllability to multi-agent controllability, and this generalized role requires the TPJ in addition to the striatum of single-agent controllability to integrate both self- and other-related information. Finally, we identified an innate positive bias toward the self during the multi-agent controllability inference, which facilitated behavioral adaptation under volatile controllability. Furthermore, low positive bias and high negative bias were associated with increased daily feelings of guilt. Our results provide a mechanism of how our sense of controllability fluctuates due to other people in our lives, which might be related to social learned helplessness and depression.

Despite recent advances in neural text generation, encoding the rich diversity in human language remains elusive.We argue that the sub-optimal text generation is mainly attributable to the imbalanced token distribution, which particularly misdirects the learning model when trained with the maximumlikelihood objective.As a simple yet effective remedy, we propose two novel methods, F 2 -Softmax and MefMax, for a balanced training even with the skewed frequency distribution.MefMax assigns tokens uniquely to frequency classes, trying to group tokens with similar frequencies and equalize frequency mass between the classes.F 2 -Softmax then decomposes a probability distribution of the target token into a product of two conditional probabilities of (i) frequency class, and (ii) token from the target frequency class.Models learn more uniform probability distributions because they are confined to subsets of vocabularies.Significant performance gains on seven relevant metrics suggest the supremacy of our approach in improving not only the diversity but also the quality of generated texts.

Due to the complexity of emotional features, there has been limited success in emotional voice conversion. One major challenge is that conversion between more than two kinds of emotions often accompanies distortion of voice signal.The factorized hierarchical variational autoencoder (FHVAE) [1] was previously shown to have an ability, called sequence-level regularization, to generate disentangled representations of both sequence-level (such as speaker identity) and segment-level features. This study exploits the FHVAE pipeline to produce disentangled representations of emotion, making it possible to greatly facilitate emotional voice conversion.We propose three versions of algorithms for improving the quality of disentangled representation and audio synthesis. We conducted three mean opinion score (MOS) surveys to assess the performance of our models in terms of 1) speaker’s voice preservation, 2) emotion conversion, and 3) audio naturalness.

This paper presents a novel approach to improve the performance of polyphonic sound event detection that combines a convolutional bidirectional recurrent neural network (CBRNN) with transfer learning. The ordinary convolutional recurrent neural network (CRNN) is known to suffer from a vanishing gradient problem, which significantly reduces the efficiency of information transfer to past events. To resolve this issue, we combine forward and backward long short-term memory (LSTM) modules and demonstrate that they complement each other. To effectively deal with the issue of overfitting that arises from increased model complexity, we apply transfer learning with a dataset that contains synthesized artifacts. We show that the model achieves faster and better performance with less data. Simulations with the 2016 TUT dataset show that the performance of the CBRNN with transfer learning is dramatically improved compared to the ordinary CRNN; the F1 score was 28.4% higher, and the error rate was 0.42 lower.

Recent attention-based end-to-end speech synthesis from text systems have achieved human-level performance. However, many approaches cause a sequence-to-sequence model to generate only averaged results of the input text, making it difficult to control the duration of utterance. In this study, we present a novel mechanism for phonemic-level duration control (PDC) in a nearly end-to-end manner in order to solve this problem. We used a teacher attention alignment generated by an annotation speech analyzer program. Our method is inspired by the idea that the duration of a phoneme is highly related to its phonemic features. These phonemic features are saved on the attention alignment by adding duration embedding to it. This enables the model to learn and control the phonemic and rhythmic features of speech. We also show that providing alignment information as a teacher loss term improves training speed and notably, makes the model better at controlling the speed of dramatic change in phonemic-level duration with subjective demonstration. As a result, we show that our PDC speech synthesis with alignment loss outperforms other baseline methods without losing the ability to control the duration of phonemes in extremely adjusted environments with faster convergence.

Recent studies in decision neuroscience revealed that two different strategies guide trial-by-trial choice behavior during reinforcement learning (RL): a goal-directed strategy and a habitual learning strategy. An increasing number of studies have provided evidence for neural substrates underlying respective strategies, suggesting that a single RL algorithm cannot make accurate predictions about individual choices. Despite rapid progress in predicting humans intentions such as movements or choices, there has been no attempt to read out human learning strategies. We proposed a novel brain-computer interface (BCI) framework for decoding human learning strategies from electroencephalography (EEG) data. To circumvent the issue that there is no gold standard for labeling learning strategies, we trained the proposed framework classifier to best match predictions of the computational model of the neural process underlying human learning strategies. The simulation used a 1D, 2D, and 3D convolutional neural network (CNN) on 18 subjects; the EEG data demonstrated that the proposed framework successfully read out human learning strategies with very high accuracy (98.4%). Subsequently, we examined whether those learning strategies labeled from the computational model exhibited distinctive EEG patterns. We used class activation mapping (CAM) for visualization, and we identified distinctive strategy-dependent patterns in the EEG feature space. We argue that the proposed framework has great potential for decoding high-level cognitive states.

Recent evidence in neuroscience and psychology suggests that a single reinforcement learning (RL) algorithm only accounts for less than 60% of the variance of human choice behavior in an uncertain and dynamic environment, where the amount of uncertainty in state-action-state transitions drift over time. The prediction performance further decreases when the size of the state space increases. We proposed a hierarchical context-dependent RL control framework that dynamically exerted control weights on model-based (MB) and multiple model-free (MF) RL strategies associated with different task goals. To properly assess the validity of the proposed method, we considered a two-stage Markov decision task (MDT) in which the three different types of context changed over time. We trained 57 different RL control models on a Caltech MDT data set; then, we assessed their prediction performance using a Bayesian model comparison. This large-scale computer simulation analysis revealed that the model providing the most accurate prediction was the version that implemented the competition between the MB and multiple goal-dependent MF RL strategies. The present study demonstrates the applicability of the goal-driven RL control to a variety of real-world human-robot interaction scenarios.

The imbalanced data problem refers to a skewed distribution of data over each class. Learning with imbalanced data constitutes a significant challenge for both research and industry applications. This often impairs performance of even powerful machine learning algorithms, such as Deep Artificial Neural Networks (DNN). To alleviate this problem, this paper proposes a neuroscience-inspired approach to train DNN, in which each neuron learns to filter the input while minimizing the training error though error back propagation. The proposed framework possesses two important structural and functional characteristics of biological vision systems: (1) each layer of the visual hierarchy has a non-uniform receptive field distribution and (2) the visual cortex learns to control its attention during development. The learned filter makes each neuron react only to a specific input range, encouraging some neuron committed to learn imbalanced samples. We demonstrate that the proposed model achieves improved performance in various imbalanced data-learning scenarios.

Social norms have a potential to contribute to advances in social intelligence. One approach to take this advantage in the design of virtual agents is the use of institutional models - a social reasoning framework which brings about social norms - in conjunction with classical AI techniques, to achieve the appropriate recognition of complex situations and provide guidance on the subsequent choice of adequate action(s) with norms. In this paper, we aim to show that the combination of an institution providing social reasoning and BDI agents providing individual reasoning, establishes a framework for socially intelligent behaviour by the interplay between: (i) the institution and Intelligent Virtual Agents (IVAs), and (ii) norms maintained by the institution and the mental states of IVAs. From an engineering point of view, the framework provides a separation of concerns because the BDI agent is augmented with the capacity to process social obligations, while the specification and verification of social structure resides in the institutional models. We illustrate our approach with two scenarios: one on queuing and another on inter-personal distance theory.

Electroencephalogram (EEG) signals exhibit highly irregular patterns. This irregularity, which arises from i.i.d. measurement noise, has been partially resolved by memoryless classifiers, such as deep convolutional neural networks (CNN). However, there are other major sources of irregularity, including brain network modes, mental states, and various physiological factors. These internal states drift over time, in which case it would be better to use memory-based neural networks, such as long short-term memory networks (LSTM). This paper presents a novel EEG signal classification framework that resolves a trade-off between memoryless and memory-based classification. The proposed method uses deep reinforcement learning (RL) to find a trial-by-trial control strategy for the attention control system that switches between CNN (memoryless) and LSTM (memory-based)-or is a mixture of both. The simulation on the EEG dataset, which was collected while performing a complex cognitive task, shows that the proposed attention control system outperforms other EEG classification methods.

Knowledge-bases (KB) are usually incomplete due to an exponential increase in the amount of data and its high-order dependency. This fuels a strong demand for KB completion. This paper presents a novel automated KB completion framework that performs the following process cycle: (i) exploring missing factors, (ii) querying the incomplete knowledge, (iii) reasoning on relations between newly discovered factors (iv) and updating the KB. The proposed framework uses the combination of collaborative filtering and deep reinforcement learning. First, it uses memory-based collaborative filtering to infer the missing factors by identifying an head entity and its association with a missing triplet. It then carries out multi-hop relation reasoning using deep reinforcement learning to complete the KB. Simulations on two public datasets demonstrate that our framework successfully completes the KB with high precision without any prior knowledge or additional information.

Electromyogram (EMG) signal-based gait phase recognition for walking-assist devices warrants much attention in human-centered system design as it well exemplifies human-in-the-loop control where the system's prediction directly affects subsequent walking motion. Since walking motion poses considerable variations in electrode placement, performance reliability of such systems is contingent on a combination of electrode montage and a feature extraction method that takes into account underlying physiological factors of peripheral muscles where electrodes are placed. In many practical applications, however, proper consideration of effects of the electrode location variation on performance reliability of the system has received scant empirical attention. Here, based on a user-centered design principle, we establish a gait phase recognition system that is capable of rigidly controlling ill effects due to this covariate by carrying out a large-scale analysis that combines statistical, model-based, and empirical approaches. In doing so, we have developed a special sensing suit for the control of electrode placement and a reliable data acquisition. We then have conducted a nonparametric statistical analysis on class separability values of thirty types of EMG feature sets, followed by a model-based analysis to address the tradeoff between class separability and dimensionality. To further address the issue of how these results generalize to independent systems and data sets, we have carried out an empirical performance assessment over six classification methods. First, the two feature types, Integral of Absolute Value and Histogram, and a combination of the two are shown to be robust against electrode location variations while providing a firm performance guarantee. Second, system organization scenarios are presented on a case-by-case basis, allowing us to trade off system complexity for on-line adaptation capability. Collectively, our integrated analysis lends itself to formulating a guideline for design of highly reliable EMG signal-based walking assistant systems in a variety of smart home scenarios.

In designing autonomous service systems such as assistive robots for the aged and the disabled, discovery and prediction of human actions are important and often crucial. Patterns of human behavior, however, involve ambiguity, uncertainty, complexity, and inconsistency caused by physical, logical, and emotional factors, and thus their modeling and recognition are known to be difficult. In this paper, a nonsupervised learning framework of human behavior patterns is suggested in consideration of human behavioral characteristics. Our approach consists of two steps. In the first step, a meaningful structure of data is discovered by using Agglomerative Iterative Bayesian Fuzzy Clustering (AIBFC) with a newly proposed cluster validity index. In the second step, the sequence of actions is learned on the basis of the structure discovered in the first step and by utilizing the proposed Fuzzy-state Q--learning (FSQL) process. These two learning steps are incorporated in an amalgamated framework, AIBFC-FSQL, which is capable of learning human behavior patterns in a nonsupervised manner and predicting subsequent human actions. Through a number of simulations with typical benchmark data sets, we show that the proposed learning method outperforms several well-known methods. We further conduct experiments with two challenging real-world databases to demonstrate its usefulness from a practical perspective.

In this brief, we consider kernel methods for classification (Shawe-Taylor and Cristianini, 2004) from a separability point of view and provide a representation of the Fisher criterion function in a kernel feature space. We then show that the value of the Fisher function can be simply computed by using averages of diagonal and off-diagonal blocks of a kernel matrix. This result further serves to reveal that the ideal kernel matrix is a global solution to the problem of maximizing the Fisher criterion function. Its relation to an empirical kernel target alignment is then reported. To demonstrate the usefulness of these theories, we provide an application study for classification of prostate cancer based on microarray data sets. The results show that the parameter of a kernel function can be readily optimized.

Memory impaired people, e.g., dementia people, requires careful social support. Dementia people are getting increased with very high rate especially. It has been reported that regular daily life can alleviate the symptom of the memory loss. Accordingly, human behavior suggestion is highly expected to help memory impaired people live regularly. In this paper, we propose a human behavior suggestion system based on fuzzy-state Q-learning for memory impaired person, and show its possible application in intelligent sweet home. Specifically, we claim that an averaged frequency feature is an important factor. In order to evaluate the validity of the proposed human behavior suggestion system, we conduct experiments with a real world data set, INT DB. The experimental results show that the proposed system with the averaged frequency feature outperforms the existing system.

This paper conveys Bayesian interpretation of improved integrated adaptive fuzzy clustering(IAFC), which is one of the adaptive fuzzy neural network models and suggests upper bound of vigilance parameter, which gives us a guideline to endow IAFC with flexibility within the framework of minimum risk classifier. Besides, we proposed the off-line and on-line learning strategy of IAFC. The proposed techniques are applied to construct facial expression recognition system dealing with neutral, happy, sad, and angry. We empirically show that proposed methods are able to outperform the conventional IAFC.

The facial expression recognition (FER) is one of the biosignal-based recognition techniques which attract a lot of attention recently. To deal with its complex characteristics effectively, we adopt the soft computing techniques (SCT) such as fuzzy logic, neural networks, genetic algorithm and/or rough set technique. In this paper, we overview the state-of-the-art reports on FER in view of SCT, and introduce some interesting works done by our group on the SCT-based facial emotional expression recognition. Specifically, 1) fuzzy observer-based approach, 2) personalized FER system based on fuzzy neural networks, and 3) Gabor wavelet neural networks are briefly discussed