Voice conversion (VC) is a task that alters the voice of a person to suit different styles while conserving the linguistic content. Previous state-of-the-art technology used in VC was based on the sequence-to-sequence (seq2seq) model, which could lose linguistic information. There was an attempt to overcome this problem using textual supervision; however, this required explicit alignment, and therefore the benefit of using seq2seq model was lost. In this study, a voice converter that utilizes multitask learning with text-to-speech (TTS) is presented. By using multitask learning, VC is expected to capture linguistic information and preserve the training stability. This method does not require explicit alignment for capturing abundant text information. Experiments on VC were performed on a male-Korean-emotional-text-speech dataset to convert the neutral voice to emotional voice. It was shown that multitask learning helps to preserve the linguistic content.

We propose an approach for digitally altering people's outfits in images. Given images of a person and a desired clothing style, our method generates a new clothing item image. The new item displays the color and pattern of the desired style while geometrically mimicking the person's original item. Through superimposition, the altered image is made to look as if the person is wearing the new item. Unlike recent works with full-image synthesis, our work relies on segment synthesis, yielding benefits in virtual try-on. For the synthesis process, we assume two underlying factors characterizing clothing segments: geometry and style. These two factors are disentangled via preprocessing and combined using a neural network. We explore several networks and introduce important aspects of the architecture and learning process. Our experimental results are three-fold: 1) on images from fashion-parsing datasets, we demonstrate the generation of high-quality clothing segments with fine-level style control; 2) on a virtual try-on benchmark, our method shows superiority over prior synthesis methods; and 3) in transferring clothing styles, we visualize the differences between our method and neural style transfer.

Many speech enhancement methods try to learn the relationship between noisy and clean speechs, obtained using an acoustic room simulator. We point out several limitations of enhancement methods relying on clean speech targets; the goal of this letter is to propose an alternative learning algorithm, called acoustic and adversarial supervision (AAS). AAS makes the enhanced output both maximizing the likelihood of transcription on the pre-trained acoustic model and having general characteristics of clean speech, which improve generalization on unseen noisy speeches. We employ the connectionist temporal classification and the unpaired conditional boundary equilibrium generative adversarial network as the loss function of AAS. AAS is tested on two datasets including additive noise without and with reverberation, Librispeech + DEMAND, and CHiME-4. By visualizing the enhanced speech with different loss combinations, we demonstrate the role of each supervision. AAS achieves a lower word error rate than other state-of-the-art methods using the clean speech target in both datasets.

In this letter, we propose an N-best rescoring system that integrates attentional information for locally confusing words extracted from alternative hypotheses to a conventional speech recognition system. The attentional information is derived by adapting a test input feature for the word of interest, which is motivated by the top-down selective attention mechanism of the brain. To rescore the competing hypotheses, we define a new confidence measure that contains both the conventional posterior probability and the attentional information for the confusing words. In addition, a neural network is designed to provide different weights within the confidence measure for each utterance. The network is then optimized to minimize the word error rates. Tests on the Wall Street Journal and Aurora4 speech recognition tasks were conducted, and our best results achieve a word error rate of 3.83% and 11.09%, yielding a relative reduction of 5.20% and 2.55% over baselines, respectively.

Convolutional neural networks (CNNs) with convolutional and pooling operations along the frequency axis have been proposed to attain invariance to frequency shifts of features. However, this is inappropriate with regard to the fact that acoustic features vary in frequency. In this paper, we contend that convolution along the time axis is more effective. We also propose the addition of an intermap pooling (IMP) layer to deep CNNs. In this layer, filters in each group extract common but spectrally variant features, then the layer pools the feature maps of each group. As a result, the proposed IMP CNN can achieve insensitivity to spectral variations characteristic of different speakers and utterances. The effectiveness of the IMP CNN architecture is demonstrated on several LVCSR tasks. Even without speaker adaptation techniques, the architecture achieved a WER of 12.7% on the SWB part of the Hub5'2000 evaluation test set, which is competitive with other state-of-the-art methods.

From electroencephalography (EEG) data during self-relevant sentence reading, we were able to discriminate two implicit intentions: 1) "agreement" and 2) "disagreement" to the read sentence. To improve the classification accuracy, discriminant features were selected based on Fisher score among EEG frequency bands and electrodes. Especially, the time-frequency representation with Morlet wavelet transforms showed clear differences in gamma, beta, and alpha band powers at frontocentral area, and theta band power at centroparietal area. The best classification accuracy of 75.5% was obtained by a support vector machine classifier with the gamma band features at frontocentral area. This result may enable a new intelligent user-interface which understands users' implicit intention, i.e., unexpressed or hidden intention.

We propose a novel experiment paradigm to measure human trust on machine during a collaborative and egoistic theory-of-mind game. To show a different level of human trust on machine partners, we control the technical capability and humanlike cues of the autonomous agent in the cognitive experiments while recording participant's electroencephalography (EEG). The measured human trust values at various situations will be used to develop a dynamic trust model for efficient human-machine systems.

Information and communication technologies supply data every day at incredibly increasing rate, however, almost all of the accumulated data are unlabeled and obtaining their labels is expensive and time-consuming. Among the raw data, selecting and labeling some samples expected to be more informative than others can enhance machines without high cost. This process is called selective sampling, essential part of active learning. So far, most researches have concentrated on classical uncertainty measures to acquire informative data, which is related to "exploitation" process of learning. However, when the initial labeled dataset is too small or biased, the early stage model can be unreliable and its decision boundary would be over-fitted to the initial data. Moreover, the obtained data by the exploitation strategy may exacerbate the model further. We introduced "exploration" strategy as well as "exploitation" strategy. In this paper, we employ Self-Organizing Maps (SOM), one of neural networks to estimate and explore data distribution. For exploitation, margin sampling is applied to the classifier, neural network with soft-max output layer. The effectiveness proposed methods are demonstrated on ILSVRC-2011 image classification task based on features extracted from well-trained Convolutional Neural Networks (CNN). Active learning with exploration strategy shows its potential by stabilizing the early stage model and reducing the classification error rate, and finally making it to be high-quality models.

For the successful interaction between human and machine agents, the agents need understand both explicitly-presented human intention and unpresented human mind. Although the current human-agent interaction (HAI) systems mainly rely on the former with keystrokes, speech, and gestures, the latter will play an important role for the new and up-coming HAIs. In this talk we will present our continuing efforts to understand unpresented human mind, which may reside at the internal states of neural networks in human brain and may be estimated from brain-related signals such as fMRI (functional Magnetic Resonance Imaging), EEG (Electroencephalography), and eye movements. We hypothesized that the space of brain internal states have several independent axes, of which temporal dynamics have different time scales. Special emphasis was given to human memory, trustworthiness, and sympathy to others during interactions. Human memory changes much slowly in time, and is different from person to person. Therefore, by analyzing brain-related signals from many stimulating images, it may be possible to identify a person. On the other hand the sympathy to others has much shorter time constants during human-agent interactions, and may be identified for each user interaction. The trustworthiness to others may have slightly longer time constants, and may be accumulated by temporal integration during sequential interactions. Therefore, we measured brain-related signals during sequential Theory-of-Mind (ToM) games. Also, the effects of human-like cues of the agents to the trustworthiness were evaluated. At this moment the estimation of human internal states utilizes brain-related signals such as fMRI, EEG, and eye movements. In the future the classification systems of human internal states will be trained with audio-visual signals only, and the current study will provide near-ground-truth labels.

Implicit intention is the intention that is not expressed externally but having in one's mind. Implicit intention is difficult to be recognized, but it can be significant information if it is recognized with some measures. When people buy something, they also have implicit intention in their mind, whether I buy this or not. We proposed an experimental paradigm to recognize shopper's implicit intention, and the result of experiment was analyzed in this paper. On the experiment, subjects were instructed to select items to buy from the candidates, and eye-tracking and speech data were recorded during the selection. On data analysis, measures discriminating the existence of implicit shopping intention were selected and compared. From the result, fixation duration, fixation count, multiplication of first fixation duration, and visit count showed different tendency between two cases: when people have intention to buy it and when people do not have. By using this standards, implicit shopping intention of people can be recognized.

We present a pattern recognition framework to improve committee machines of deep convolutional neural networks (deep CNNs) and its application to static facial expression recognition in the wild (SFEW). In order to generate enough diversity of decisions, we trained multiple deep CNNs by varying network architectures, input normalization, and weight initialization as well as by adopting several learning strategies to use large external databases. Moreover, with these deep models, we formed hierarchical committees using the validation-accuracy-based exponentially-weighted average (VA-Expo-WA) rule. Through extensive experiments, the great strengths of our committee machines were demonstrated in both structural and decisional ways. On the SFEW2.0 dataset released for the 3rd Emotion Recognition in the Wild (EmotiW) sub-challenge, a test accuracy of 57.3% was obtained from the best single deep CNN, while the single-level committees yielded 58.3% and 60.5% with the simple average rule and with the VA-Expo-WA rule, respectively. Our final submission based on the 3-level hierarchy using the VA-Expo-WA achieved 61.6%, significantly higher than the SFEW baseline of 39.1%.

Although Nonnegative Matrix Factorization (NMF) has been widely known as an effective feature extraction method, which provides part-based representation and good reconstruction, there were relatively few researches using NMF for color image processing. Particularly, many studies are now using Convolutional Neural Network (CNN) in combined with Auto-Encoder (AE) or Restricted Boltzmann Machine (RBM) for learning features of color images. In this paper, we explore the ability of NMF to handle color images. Especially, a new method using NMF to learn features in CNN is proposed. In our experiments conducted on CIF ARIO, NMF shows the feasibility for reconstruction and classification of color images. Furthermore, unlike edge- or curve- shaped features learned by AE and RBM in CNN, our method provides dot- shaped features. These new types of features could be considered as basic building blocks in the lowest level of constructing images. Our results demonstrate that NMF is capable of being a supporting tool for CNN in learning features.

With the increasing importance of smart gadgets in our daily lives, there is a need for an automatic piano transcription system in various multimedia services. For automatic piano transcription, the string inharmonicity coefficient (B) and fundamental frequency (f0) should be detected robustly and accurately. The proposed triplet-sequentially additive partial (SAP) algorithm improves the current B estimation algorithm in terms of both performance and speed with less prior knowledge. Additionally, this joint (B, f0) estimation algorithm is applied directly to the transcription of real piano recordings, and the 4.41% improvement of accuracy was achieved over another transcription system that had both similar processing steps and feature extraction method.

The authors propose a novel object detection algorithm based on shape matching using a single sketch of an object. The proposed algorithm uses circular arc segments to describe image edges; this approach is advantageous for shape description, shape expression and reconstruction. Circular arcs are initially segmented from the image contour using the split‐and‐merge method, and they are extended, being partially overlapped with neighbouring circular arcs. The extracted circular arcs of the object sketch constitute an attributed relational graph as a structured object model. Circular arcs in the test image are refined by the bottom‐up process of circular arc extension, and matched with circular arcs in the object model by the top‐down process of end‐point adjustment. The authors design end‐point‐based shape descriptors to encode local shape information. Hough voting aggregates the detection of circular arcs to localise the object. Probabilistic relaxation verifies the detection candidates and delineate the object boundaries. The proposed object detection system benefits from reliable extraction of contour segments, efficient and discriminative shape encoding, and flexible and robust shape matching. It exhibits competitive object detection performance in experiments using real images.

The objective of human computer interface (HCI) research is to make machine that understands human intention with high accuracy. In the future, machine will be able to understand human intention and communicate with human even though there is no explicit expression such as speech, gesture, etc. In this study, a new experiment design is proposed to understand human intention by using electroencephalography (EEG). Two types of stimulus are given to the subjects; affirmative and negative sentences. Each sentence was separated by two blocks which are contents and sentence end. Subjects were asked to make a decision of agreement or disagreement after sentence end is shown. Based on the EEG analysis, intention to the sentences was found while seeing contents, i.e., before giving an explicit answer. It is shown that decision making is done before the end of the sentence, and answering “Yes” or “No” is determined only up to the sentence end. This study shows the relationship of implicit intention and average event-related potentials (ERPs) at the frontal sites and how to predict implicit intention based on the central frontal ERPs using support vector machine (SVM).

How the cognitive capabilities of humans are mapped on the underlying computational framework is far to be completely understood. Despite it is an open problem, some evidences show that the interaction among the cortex, the amygdala and the thalamus is involved in the composition of elementary behaviours to solve complex tasks. The cortex is well-known to have a certain degree of plasticity to categorise the processed information whereas the other two brain areas are involved in the association of an emotional state to the incoming sensory information. In the specific, the amygdala is involved in the generation of innate affective states whereas the thalamus deals with the developing of new goals. The interaction among these three brain structures plays an important role in the emerging of a middle level of cognition. We claim that the cognitive model could fill the gap between the high-level reasoning and low level sensory-motor coordination. In this paper, firstly we model the above mentioned brain areas and second we investigate the interaction of these areas, producing a goal-generating architecture. Finally, the goal-generating system is tested on a NAO robot, in order to validate and extend some previously published results.

We present a new unified training scheme using a feature extractor and HMM classifiers for better speech recognition performance. Both feature extractor and classifier are trained simultaneously to minimize classification error. Multiframe features are extracted using spectro-temporal dynamics and the feature extractor is implemented as a multilayer network, which is trained by a backpropagation (BP) algorithm with the help of an HMM inversion algorithm. The initial parameter values of the feature extractor are set for Mel-frequency cepstral coefficients (MFCC) as well as their delta and acceleration components. The experiments for phoneme classification demonstrate the practicality of unified training.

This letter proposes a computational auditory scene analysis (CASA) model for monaural speech separation. In this model, we integrate three biologically inspired approaches for: auditory spectrogram generation, analysis of its spectro-temporal content, and tracking its harmonic structure. In a top-down process, the estimated ideal binary mask (EIBM) is calculated using the spectral amplitude of the extracted spectrograms to enhance the harmonic filters for separation. Experimental results showed that our model outperformed the harmonic magnitude suppression technique in both signal-to-interference ratio and percentage of crosstalk. Moreover, the result is comparable with a current state-of-the-art system.

This paper proposes a new Computational Auditory Scene Analysis (CASA) approach based on a 2D spectro-temporal analysis and harmonic separation. The 2D processing, socalled Grating Compression Transform (GCT), analyzes the spectro-temporal content of the spectrogram, mimicking the processing of the primary auditory cortex. The estimated pitches from the GCT analysis are used for separation using harmonic magnitude suppression (HMS). A powerful aspect of our model is requiring no prior training on a specific training corpus. A baseline system based on the harmonic separation is designed for comparison. Since the baseline system is similar to the proposed except the auditory-cortex-like analysis, the SIR results illustrate its importance in this task.

This letter proposes a new method to realize a nonlinear mapping of one-to-many correspondences. Assuming that a small number of training pairs are given with their actual correspondences, each tangent space is locally constructed on a submanifold around each labeled sample. Moreover, the linear transformation between paired tangent spaces is derived by solving an optimization problem, which is designed to bring locally linear maps into closer proximity in each class. Finally, a global nonlinear mapping is realized by combining these locally linear maps. In simulations of an S-curve to Swiss-roll, a lip to speech, and room impulse response to position of microphone mappings, the proposed method shows the remarkable mapping ability.

A conventional linear model based on Negentropy maximization extracts statistically independent latent variables which may not be optimal to give a discriminant model with good classification performance. In this paper, a single-stage linear semisupervised extraction of discriminative independent features is proposed. Discriminant independent component analysis (dICA) presents a framework of linearly projecting multivariate data to a lower dimension where the features are maximally discriminant with minimal redundancy. The optimization problem is formulated as the maximization of linear summation of Negentropy and weighted functional measure of classification. Motivated by independence among extracted features, Fisher linear discriminant is used as the functional measure of classification. Experimental results show improved classification performance when dICA features are used for recognition tasks in comparison to unsupervised (principal component analysis and ICA) and supervised feature extraction techniques like linear discriminant analysis (LDA), conditional ICA, and those based on information theoretic learning approaches. dICA features also give reduced data reconstruction error in comparison to LDA and ICA method based on Negentropy maximization.

This letter looks at the physics behind sensory data by identifying the parameters that govern the physical system and estimating them from sensory observations. We extend Takens's delay-embedding theorem to the dynamical systems controlled by parameters. An embedding of the product space of the phase and the parameter spaces of the dynamical system can be obtained by the delay-embedding map, provided that the parameter of the dynamical system changes slowly. The reconstruction error is bounded for slowly varying parameters. A manifold learning technique is applied to the embedding obtained to extract a low-dimensional global coordinate system representing the product space. The phase space of the deterministic dynamics can be contracted by using the adjacency relationship in time, which enables recovery of only the parameter space. As examples, the manifolds of synthetic and real-world vowels with time-varying fundamental frequency (F(0)) are analyzed, and the F(0) contours are extracted by an unsupervised algorithm. Experimental results show that the proposed method leads to robust performance under various noise conditions and rapid changes of F(0) compared with the current state-of-the-art F(0) estimation algorithms.

A cluster of workstations may be employed for reducing fault simulation time greatly. Fault simulation can be parallelized by partitioning fault list, the test vector or both. In this study, parallel fault simulation algorithm called PAUSIM has been developed by parallelizing AUSUM which consists of logic simulation and two steps of fault simulation for sequential logic circuits. Compared to other algorithms, PAUSIM-CY avoids redundant work by a judicious task decomposition. Also, it adopts a cyclic fault partitioning method based on the LOG partitioning and local redistribution, resulting in a well-balanced load distribution. The results from the parallel implementation using MPI show a significant speed-up by PAUSIM-CY over other existing parallel algorithms.

BACKGROUND: Nonnegative matrix factorization (NMF) is a feature extraction method that has the property of intuitive part-based representation of the original features. This unique ability makes NMF a potentially promising method for biological sequence analysis. Here, we apply NMF to fold recognition and remote homolog detection problems. Recent studies have shown that combining support vector machines (SVM) with profile-profile alignments improves performance of fold recognition and remote homolog detection remarkably. However, it is not clear which parts of sequences are essential for the performance improvement. RESULTS: The performance of fold recognition and remote homolog detection using NMF features is compared to that of the unmodified profile-profile alignment (PPA) features by estimating Receiver Operating Characteristic (ROC) scores. The overall performance is noticeably improved. For fold recognition at the fold level, SVM with NMF features recognize 30% of homolog proteins at > 0.99 ROC scores, while original PPA feature, HHsearch, and PSI-BLAST recognize almost none. For detecting remote homologs that are related at the superfamily level, NMF features also achieve higher performance than the original PPA features. At > 0.90 ROC50 scores, 25% of proteins with NMF features correctly detects remotely related proteins, whereas using original PPA features only 1% of proteins detect remote homologs. In addition, we investigate the effect of number of positive training examples and the number of basis vectors on performance improvement. We also analyze the ability of NMF to extract essential features by comparing NMF basis vectors with functionally important sites and structurally conserved regions of proteins. The results show that NMF basis vectors have significant overlap with functional sites from PROSITE and with structurally conserved regions from the multiple structural alignments generated by MUSTANG. The correlation between NMF basis vectors and biologically essential parts of proteins supports our conjecture that NMF basis vectors can explicitly represent important sites of proteins. CONCLUSION: The present work demonstrates that applying NMF to profile-profile alignments can reveal essential features of proteins and that these features significantly improve the performance of fold recognition and remote homolog detection.

A high-performance fundamental frequency estimator inspired by the human auditory system is presented. It uses a cubic-root transform and contrast enhancement on the spectral domain to enhance weak harmonics and suppress noise. The proposed weighted unbiasing process reduces multiple or submultiple pitch errors efficiently. Evaluation is performed under the presence of white noise, babble noise, and telephone channel distortion. The results show that the proposed method performs equivalent to or better than previous state-of-the-art methods in the majority of the noise and channel conditions.

Heterogeneity has been considered in scheduling, but without taking into account the temporal variation of completion times of the sub-tasks for a divisible, independent task. In this paper, the problem of scheduling multiple, divisible independent tasks on a heterogeneous distributed computing system are addressed. The "stochastic" approach, which was previously applied to DAG scheduling, is employed for scheduling a group of multiple divisible as well as whole independent tasks. It explicitly considers the standard deviations (temporal heterogeneity) in addition to the mean execution times in deriving a schedule, in order to model more closely what would actually happen "on average" on a temporally heterogeneous system (instead of approximating the random weights by their means only as in other approaches). Through an extensive computer simulation, it has been shown that the proposed approach can improve schedules significantly over those by a scheme which uses the average weights only.

Simulated annealing (SA) is a general-purpose optimization technique widely used in various combinatorial optimization problems. However, the main drawback of this technique is a long computation time required to obtain a good quality of solution. Clusters have emerged as a feasible and popular platform for parallel computing in many applications. Computing nodes on many of the clusters available today are temporally heterogeneous. In this study, multiple Markov chain (MMC) parallel simulated annealing (PSA) algorithms have been implemented on a temporally heterogeneous cluster of workstations to solve the graph partitioning problem and their performance has been analyzed in detail. Temporal heterogeneity of a cluster of workstations is harnessed by employing static and dynamic load balancing techniques to further improve efficiency and scalability of the MMC PSA algorithms.

In this paper, the feasibility of multiple targets localization with a ultrasonic transmitter and only two ultrasonic sensors are tested and its results are given. For this purpose, an experimental hardware system and algorithm which models after bat's vocalization and auditory pathway are built. To mimic bat's behavior features, bat's chirp composed of single band/full band signal component and frequency response in bat's cochlear are modeled after. Time delay estimation is made by cross correlation and deconvolution between transmitting signal and its echoes. As a result, the combination of full band signal and filterbank shows the best performance and the reason is analyzed The physical and theoretical constraints of this approach for multiple targets localization are analyzed and some possible solutions for the improvement are proposed.

A modification is proposed to the independent component analysis (ICA)-based filterbank approach in consideration to its structural similarity with binaural auditory model of sound source localization. The estimated sound locations provide an additional cue to the learning algorithm, which is utilized for initialization and imposition of directional constraints on the subband separation networks. Directionally constrained filterbank ICA (DC-FBICA) gives faster convergence and improves separation performance for noisy mixtures having significant spectral overlap among the convolved mixture and the corrupting noise. However, only slight improvement in separation performance is observed when the additive noise is a low frequency noise, although faster convergence is still observed.

For noise-robust speech recognition, we incorporated a top-down attention mechanism into a hidden Markov model classifier with Mel-frequency cepstral coefficient features. The attention filter was introduced at the outputs of the Mel-scale filterbank and adjusted to maximize the log-likelihood of the attended features with the attended class. A low-complexity constraint was proposed to prevent the attention filter from over-fitting, and a confidence measure was introduced on the attention. A classification was made to the class with the maximum confidence measure, and demonstrated 54% and 68% reduction of the false recognition rate with 15- and 20-dB signal-to-noise ratio, respectively.

Blind source separation (BSS) is a challenging problem in real-world environments where sources are time delayed and convolved. The problem becomes more difficult in very reverberant conditions, with an increasing number of sources, and geometric configurations of the sources such that finding directionality is not sufficient for source separation. In this paper, we propose a new algorithm that exploits higher order frequency dependencies of source signals in order to separate them when they are mixed. In the frequency domain, this formulation assumes that dependencies exist between frequency bins instead of defining independence for each frequency bin. In this manner, we can avoid the well-known frequency permutation problem. To derive the learning algorithm, we define a cost function, which is an extension of mutual information between multivariate random variables. By introducing a source prior that models the inherent frequency dependencies, we obtain a simple form of a multivariate score function. In experiments, we generate simulated data with various kinds of sources in various environments. We evaluate the performances and compare it with other well-known algorithms. The results show the proposed algorithm outperforms the others in most cases. The algorithm is also able to accurately recover six sources with six microphones. In this case, we can obtain about 16-dB signal-to-interference ratio (SIR) improvement. Similar performance is observed in real conference room recordings with three human speakers reading sentences and one loudspeaker playing music

We propose a novel approach to the blind source separation (BSS) that exploits frequency dependencies within a source. In contrast to conventional algorithms that separate the sources independently in each frequency bin, we assume that dependencies exist between frequency bins in a source signal. In this manner, we can reduce or eliminate the well-known frequency permutation problem. We derive the learning algorithm by defining a cost function as an extension of mutual information between multivariate random variables and by introducing a source prior that models the inherent frequency dependencies. This results in a simple form of a multivariate score function. In simulations and real recording experiments, we evaluate the performance of the proposed method and compare it against other well-known algorithms under various conditions. Our results indicate that modeling dependencies yields improved performance and robust scaling to higher number of sources and mixtures.

The paper presents a fast new algorithm for estimating the relative time delay between a microphone pair in noisy reverberant conditions. The algorithm is derived from a novel approach to this problem, which we formulate in the framework of probabilistic graphical models. We construct a probabilistic model of the microphone signals, using models of speech and noise as building blocks. The reverberation filter coefficients and the relative time delay appear as model parameters, and are estimated by our algorithm from data. The resulting delay estimate is Bayes optimal and takes into account noise and reverberation in a principled manner. We demonstrate very good performance on data from real and simulated room environments.

The features of human lip motion from video clips are extracted by principal component analysis (PCA) and independent component analysis (ICA). Unlike many other features extracted from single-frame static images or multi-frame dynamic images, we extracted the features from the differences of consecutive frames. The PCA results in global features, while local features are extracted by the ICA. The features are extracted from several consecutive multi-frame differences as well as single-frame differences. The dynamic nature of multi-frame differences is more eminent. The resulting features maybe applicable in lip-reading and synthesis of lip motion videos with text-to-speech capability.

Channel bandwidth available for a message on a communication network varies with time and link. This variation (heterogeneity) can have a significant effect on performance of an individual message and also that of the network as a whole. Therefore, it is necessary to understand effects of bandwidth heterogeneity on the network performance in order to optimally utilize a heterogeneous communication network. The ability to use such a network optimally is highly desirable in many applications such as network-based data-intensive high performance computing. The main goal of this study, as the first step toward developing effective schemes to utilize heterogeneous communication networks, is to analyze effects of temporal and spatial heterogeneity on performance of individual messages in detail via an extensive simulation, in terms of throughput, end-to-end delay, etc. Also, the problems of path selection and multipath data transfer are considered to illustrate how the analysis results may be used in the future effort of optimizing the network performance by taking channel bandwidth heterogeneity into account.

The Korean Brain Neuroinformatics Research Program would get into the 3rd phase from June 2004 for 4 years, which is regarded as the final phase of Korean Brain National Research Program started from November 1998 for 10 years sponsored by Ministry of Science and Technology. It is a joint effort of researchers from many different disciplines including neuroscience, cognitive science, mathematics, physics, electrical engineering, computer science, etc. In the 3rd phase we would like to continue our efforts on functional artificial systems for vision, auditory, inference, and behavior. Also, we would like to develop a "digital brain" to combine all the 4 functions, and an integrated demonstration system, i.e., "artificial secretary" alias "office mate" with exceptional human-like information processing capabilities. Researches on measurement and signal analysis for neuroscience are also included.

Summary form only given. Electron-beam lithography is one of the techniques used in transferring circuit patterns onto substrates. Proximity effect caused by electron scattering imposes a severe limitation on the ultimate spatial resolution attainable by e-beam lithography. Therefore, proximity effect correction is essential particularly for fine-feature, high-density circuit patterns. Proximity effect correction is very time-consuming due to intensive computation required in the correction procedure and a large size of circuit pattern data to be processed. Therefore, it is an ideal candidate for distributed computing where the otherwise-unused CPU cycles of a number of computers on a network (cluster) can be efficiently utilized. One of the characteristics of such a cluster is its heterogeneity, i.e., the available computing power varies with computer and/or time. This variation may degrade performance of distributed computing significantly. We describe a distributed implementation of proximity effect correction, and static and dynamic load balancing schemes which attempt to minimize execution time of distributed proximity effect correction by considering heterogeneity of a cluster and nonuniformity in a circuit pattern. Also, experimental results obtained on a cluster of Sun workstations shared by multiple users are presented.

We present a new approach to perform independent component analysis (ICA) for convolved mixtures. This approach is based on filter banks, and a simplified network efficiently performs ICA with decimated signals in each subband. Decimation provides much less computational complexity and faster convergence speed than the time domain approach. Furthermore, the approach does not have a performance limitation of the frequency domain approach, and it is able to select the number of filters in the filter bank regardless of reverberation. With an oversampled filter bank, adaptive parameters can be adjusted without any information of other subbands, and the approach is suitable for parallel processing. We verify the effectiveness of the filter bank approach through simulations on adaptive noise cancelling.

In the real room environment, sound source is distorted with delayed versions of itself reflected form walls. This room reverberation severely degrades the intelligibility of speech and performance of automatic speech recognition system. Blind deconvolution is to find the inverse of reverberation channel when only convolved versions of the sources are available at the receiver. However existing blind deconvolution algorithms assume that a source signal has an independent identically-distributed (IID) non-Gaussian probability density function (PDF). In this research, colored nonstationary non-IID speech signals were transformed into an IID-like signal as possible by ICA-based independence transform and the resulting signals were processed using infomax blind deconvolution algorithm for the simulated minimum-phase finite impulse response (FIR) channels. Compared to the pre-whitening method by Torkkola, the proposed method demonstrated much better performance of about 30 dB signal-to-reverberant components ratio (SRR).

An field programmable gate array (FPGA) implementation of independent component analysis (ICA) algorithm is reported for blind signal separation (BSS) and adaptive noise canceling (ANC) in real time. In order to provide enormous computing power for ICA-based algorithms with multipath reverberation, a special digital processor is designed and implemented in FPGA. The chip design fully utilizes modular concept and several chips may be put together for complex applications with a large number of noise sources. Experimental results with a fabricated test board are reported for ANC only, BSS only, and simultaneous ANC/BSS, which demonstrates successful speech enhancement in real environments in real time.

A digital chip has been developed for isolated word recognition in real-world noisy environments. By carefully comparing recognition performance and hardware implementability a modified-ZCPA model and RBF neural network model are selected for the feature extractor and classifier, respectively. The modified-ZCPA model is based on the feature extraction mechanism of the human auditory system, and demonstrated superiority for noisy speech. The RBF network has excellent OOV (out-of-vocabulary) rejection capability as well as good recognition performance. Both the feature extractor and classifier are implemented by repetition of simple operations, which result in reduction of memory operations for the full use of memory bandwidth. The chip is custom designed at logic level without any DSP core, and implemented with an FPGA with 12 MHz clock speed.

In this paper, we proposed new speech features using independent component analysis to human speeches. When independent component analysis is applied to speech signals for efficient encoding the adapted basis functions resemble Gabor-like features. Trained basis functions have some redundancies, so we select some of the basis functions by the reordering method. The basis functions are almost ordered from the low frequency basis vector to the high frequency basis vector. And this is compatible with the fact that human speech signals have much more information in the low frequency range. Those features can be used in automatic speech recognition systems and the proposed method gives much better recognition rates than conventional mel-frequency cepstral features.

In this paper we report a speaker adaptation method when a small subset of word classes is available for the adaptation. A spectral transformation approach is used to adapt to a new speaker without changing the parameters of speaker-independent recognizer. By using "judge" network the degradation of the recognition rates for non-adapted word classes is minimized, which leads to the improvement of overall word recognition rates. The pruned judge network uses less parameters, but shows better generalization capability than full connected linear judge networks. Remarkable reduction of error rates is achieved for adapted 10 words, while maintaining almost same recognition rates for the non-adapted words. The results demonstrated much better recognition rates compared to the base-line system.

We present an algorithm which compensates for the mismatches between characteristics of real-world problems and assumptions of independent component analysis algorithm. To provide additional information to the ICA network, we incorporate top-down selec(cid:173) tive attention. An MLP classifier is added to the separated signal channel and the error of the classifier is backpropagated to the ICA network. This backpropagation process results in estimation of expected ICA output signal for the top-down attention. Then, the unmixing matrix is retrained according to a new cost function representing the backpropagated error as well as independence. It modifies the density of recovered signals to the density appropriate for classification. For noisy speech signal recorded in real environ(cid:173) ments, the algorithm improved the recognition performance and showed robustness against parametric changes.

We propose a new frame decorrelation method for robust speech recognition in noisy environments. In most cases, signal perturbation is caused by channel distortion and additive background noise, and can be modeled as a slowly varying term in either the log spectral or the linear-spectral domains. Thus, it is effective to deemphasize slowly varying stationary components in the spectral feature domain of speech signals, which can be considered as a temporal decorrelation process. The proposed method presents a well structured high-pass filter using the decorrelation principle, and provides some significant insights into existing high-pass approaches, such as relative spectral (RASTA) processing. The performance of the proposed method was evaluated by speaker-independent isolated-word recognition experiments using the hidden Markov model (HMM). Noisy speech was simulated by adding noise sources taken from the Noisex-92 database. Experimental results showed that the proposed method was effective for speech recognition with significant noise and yielded better performance than other high-pass methods. In addition, we compared the dynamic property of the proposed filter with that of delta features. The feature obtained by the proposed method may offer most of the delta feature property.

Proposes optimal learning rates in gradient descent training of multilayer perceptrons, which are a separate learning rate for weights associated with each neuron and a separate one for assigning virtual hidden targets associated with each training pattern. That is, a hidden weight vector has two optimal learning rates, one for assigning virtual hidden targets and the other for minimizing a hidden error function proposed in the paper. Effectiveness of the proposed error function was demonstrated for handwritten digit recognition and an isolated-word recognition tasks and very fast learning convergence was obtained.

Based on biological selective attention mechanism, a new algorithm is developed to improve recognition accuracy of isolated speeches in noisy environments. The attention adaptation algorithm utilizes the gradient-descent algorithm with the error backpropagation rule. With a newly-defined error measure at the input domain after the attention adaptation, the developed algorithm demonstrated high recognition rates for isolated Korean speeches in noisy environments.

This paper presents a new approach to an auditory model for robust speech recognition in noisy environments. The proposed model consists of cochlear bandpass filters and nonlinear operations in which frequency information of the signal is obtained by zero-crossing intervals. Intensity information is also incorporated by a peak detector and a compressive nonlinearity. The robustness of the zero-crossings in spectral estimation is verified by analyzing the variance of the level-crossing intervals as a function of the crossing level values. Compared with other auditory models, the proposed auditory model is computationally efficient, free from many unknown parameters, and able to serve as a robust front-end for speech recognition in noisy environments. Experimental results of speech recognition demonstrate the robustness of the proposed method in various types of noisy environments.

This letter proposes a new error function at hidden layers to speed up the training of multilayer perceptrons (MLP's). With this new hidden error function, the layer-by-layer (LBL) algorithm approximately converges to the error backpropagation algorithm with optimum learning rates. Especially, the optimum learning rate for a hidden weight vector appears approximately as a multiplication of two optimum factors, one for minimizing the new hidden error function and the other for assigning hidden targets. Effectiveness of the proposed error function was demonstrated for handwritten digit recognition and isolated-word recognition tasks. Very fast learning convergence was obtained for MLP's without the stalling problem experienced in conventional LBL algorithms.

(e) (f) Fig. 6. Results for Lenna image: (a) the original Lenna image, (b) 3-D plot of the reconstructed surface by the proposed neural-based method, (c) 3-D plot of the reconstructed surface by the Lambertain model with the estimated light source direction by Zheng and Chellappa’s method, (d) 3-D plot of the reconstructed surface by the Lambertian model with incorrect light source direction, (e) 3-D plot of the reconstructed surface by the Torrance‐Sparrow model with the estimated light source direction by Zheng and Chellappa’s method, and (f) 3-D plot of the reconstructed surface by the Torrance‐Sparrow model with incorrect parameters setting.

In this paper, we describe how our computational model can be used for the problems of processor allocation and task mapping. The intended applications for this model include the dynamic mapping problems of shrinking or spreading an existing mapping when the available pool of processors changes during execution of the problem. The concept of problem edge class and other features of our model are developed to realistically and efficiently support task partitioning and merging for static and dynamic mapping. The model dictates realistic changes in the computation and communication characteristics of a problem when the problem partitioning is modified dynamically. This model forms the basis of our algorithms for shrinking and spreading, and yields realistic results for a variety of problems mapped onto real systems. An emulation program running on a network of workstations under PVM is used to measure execution times for the mapping solutions found by the algorithms. The results indicate that the problem edge class is a crucial consideration for processor allocation and task mapping.

A novel method is proposed to incorporate acoustic contextual information into speech recognition systems based on the hidden Markov model (HMM). Frame correlation exponents and transition costs are introduced to measure the effects of contextual information and modify maximum likelihood solutions in standard HMMs. The contextual information parameters reflect both time correlation among feature vectors and boundary effects between HMM states. Significant reduction of error rates is achieved for a continuous speech recognition task.

High throughput is required in many tasks, especially real-time applications. A logical structure of a parallel computing system for such applications is a pipeline of multiprocessor modules to be referred to as PMM. In this paper, a linear array based realization of PMM, referred to as PMMLA (PMM based on Linear Array) is proposed. The main design objective is to achieve uncompromised performance by a compact and flexible hardware structure. This paper describes the organization and operation of a PMMLA, analyzes its performance in detail, and compares it to other possible implementations theoretically and via emulation on nCUBE/2.

SPEECH RECOGNITION Doh-Suk Kimy, Jae-Hoon Jeongy, Soo-Young Leey, Rhee M. Kilz yDepartment of Electrical Engineering/ zDivision of Basic Science Korea Advanced Institute of Science and Technology 373-1 Kusong-dong, Yusong-gu, Taejon 305-701, Korea E-mail: [email protected] ABSTRACT Zero-crossings with peak amplitudes (ZCPA) model motivated by human auditory periphery is simple compared with other auditory models, but powerful speech analysis tool for robust speech recognition in noisy environments. In this paper, improvement in recognition rate of ZCPA model is addressed by incorporating time-derivative features with several di erent time-derivative window lengths. Experimental results show that ZCPA has relatively higher sensitivity to derivative window length than conventional feature extraction algorithms. Also, experimental comparisons with several front-ends including some auditorylike schemes in real-world noisy environments demonstrate the robustness of ZCPA model. ZCPA model shows superior performance compared with other frontends especially in noisy condition corrupted by white Gaussian noise.

The ensemble interval histogram (EIH) is an auditory model which can be used as a robust "front-end" for speech recognition systems. The utilization of multiple level-crossing detectors in the EIH provides frequency and intensity information, which may be useful for speech processing. Proper determination of the number of levels and the level values is very important for reliable performance of the system. An analytic relationship is developed for the variance and SNR of the level-crossing intervals as a function of the crossing level value, and a new feature extraction method based on zero-crossings with peak amplitudes is proposed for robust speech recognition in noisy environments. The proposed method not only can preserve intensity information, but also is robust to noise in estimating the frequency information without the need to determine the level values and the number of levels. Experimental results show the robustness of the proposed method.

Simulated annealing is a general-purpose optimization technique capable of finding an optimal or near-optimal solution in various applications. However, the long execution time required for a good quality solution has been a major drawback in practice. Extensive studies have been carried out to develop parallel algorithms for simulated annealing. Most of them were not very successful, mainly because multiple processing elements (PEs) were required to follow a single Markov chain and, therefore, only a limited parallelism was exploited. In this paper, we propose new parallel simulated annealing algorithms which allow multiple Markov chains to be traced simultaneously by PEs which may communicate with each other. We have considered both synchronous and asynchronous implementations of the algorithms. Their performance has been analyzed in detail and also verified by extensive experimental results. It has been shown that for graph partitioning the proposed parallel simulated annealing schemes can find a solution of equivalent (or even better) quality up to an order of magnitude faster than the conventional parallel schemes. Among the proposed schemes, the one where PEs exchange information dynamically (not with a fixed period) performs best.

The early version of recursive refinement, an algorithm, for fast task mapping is reported in this paper. An intended application for this algorithm is dynamic load balancing on a parallel/distributed system, including a network of workstations. Since this requires a fast mapping, the algorithm restricts the range of tasks movement during optimization and maps groups of tasks to groups of PE's in each iteration. Tasks are allowed to be swapped only between the two partitions that comprise the parent partition from the previous iteration. More freedom of task movement is unnecessary because structural characteristics of the problem graph are used to guide the mapping. The groups are derived from a spanning tree that is constructed from the original problem graph and is used to identify the structural characteristics required for the algorithm. Experimental results show that the algorithm is able to achieve a mapping quality as good as that by another mapping scheme characteristic of a class of algorithms that require an order of magnitude more time.

In this paper we discuss our method for simulating the exposure of circuit patterns with electron beam lithography. Our approach divides the exposure contribution at a point into two parts: global exposure, which considers circuit elements located far away from the point, and local exposure, which considers elements in close proximity to the point. Our method is significantly faster than using direct convolution, and is able to predict the result of the exposure with a high degree of accuracy. In this paper we will present our method in detail and analyse its performance through simulation and experimental results.

Hot spots are notorious for degrading the performance of a parallel algorithm. We attempt to minimize the hot-spot access time for a class of problems, namely, Large-Scale Data-Parallel (LSDP) algorithms, on a 2D mesh. An LSDP algorithm has rich data parallelism but without exclusive task and data partitioning. Our approach is to allocate the hot spots at the optimal locations such that the hot-spot access time is minimized. Also, we have designed scheduling algorithms which control hot spot access sequences to achieve the minimal access time. Both uniform and nonuniform hot spots have been considered in this study. We have analytically derived the optimal allocations for wrapped-around and non-wrapped-around square meshes. The theoretical results have been verified by parallelizing the EM algorithm for 3D PET image reconstruction on the Intel iPSC/860.>

The expectation maximization (EM) algorithm is one of the most suitable iterative methods for positron emission tomography (PET) image reconstruction; however, it requires a long computation time and an enormous amount of memory space. To overcome these problems, we present two classes of highly efficient parallelization schemes: homogeneous and inhomogeneous partitionings. The essential difference between these two classes is that the inhomogeneous partitioning schemes may partially overlap the communication with computation by deliberate exploitation of the inherent data access pattern with a multiple-ring communication pattern. In theory, the inhomogeneous partitioning schemes may outperform the homogeneous partitioning schemes. However, the latter require a simpler communication pattern. In an attempt to estimate the achievable performance and to analyze the performance degradation factors without actual implementation, we have derived efficiency prediction formulas for closely estimating the performance for the proposed parallelization schemes. We propose new integration and broadcasting algorithms for hypercube, ring, and n-D mesh topologies, which are more efficient than the conventional algorithms when the link setup time is relatively negligible. The concept of the proposed task and data partitioning schemes, the integration and broadcasting algorithms, and the efficiency estimation methods can be applied to many other problems that are rich in data parallelism, but without balanced exclusive partitioning.>

The authors explore parallel processing issues, such as relationships between speedup and problem partitioning schemes, and problem size and time duration of each iteration for PIC (particle in cell) method on different multiprocessors. A partitioning scheme, hybrid partitioning, is introduced. Hybrid partitioning has evolved out of two general approaches to PIC problem decomposition on multiprocessors, partitioning particles and partitioning the space. The authors chose the shared memory multiprocessor environment for analyzing the parallel (distributed computing) algorithms. Two different BBN Butterfly machines (GP1000 and TC2000) were employed as testbeds.>

TAG (Training by Adaptive Gain) is a new adaptive learning algorithm developed for optical implementation of large-scale artificial neural networks. For fully interconnected single-layer neural networks with N input and M output neurons TAG contains two different types of interconnections, i.e., M N global fixed interconnections and N + M adaptive gain controls. For two-dimensional input patterns the former may be achieved by multifacet holograms, and the latter by spatial light modulators (SLMs). For the same number of input and output neurons TAG requires much less adaptive elements, and provides a possibility for large-scale optical implementation at some sacrifice in performance as compared to the perceptron. The training algorithm is based on gradient descent and error backpropagation, and is easily extensible to multilayer architecture. Computer simulation demonstrates reasonable performance of TAG compared to perceptron performance. An electrooptical implementation of TAG is also proposed.

A system design/scheduling strategy is described for a real-time parallel image processing system. A parallel image processing model based on the spatial and temporal parallelisms extractable in image processing tasks is formulated. This model consists of linear pipeline stages, each of which is a multiprocessing module. The strategy is discussed for two cases: static and dynamic. The description of the detailed hardware structure of each module is not attempted because of its dependency on a specific task. In the static case, where the processing time is constant, the processing times of all the stages are adjusted to be identical. In the dynamic case, where the processing time varies with each image, the strategy needs to be modified to achieve the maximum possible processing speed. The strategy is demonstrated for the static case by implementing image processing tasks on two different multiprocessor systems.>

In this correspondence, a parallel 2-D convolution scheme is presented. The processing structure is a mesh connected array processor consisting of the same number of simple processing elements as the number of pixels in the image. For most windows considered, the number of computation steps required is the same as that of the coefficients of a convolution window. The proposed scheme can be easily extended to convolution windows of arbitrary size and shape. The basic idea of the proposed scheme is to apply the 1-D systolic concept to 2-D convolution on a mesh structure. The computation is carried out along a path called a convolution path in a systolic manner. The efficiency of the scheme is analyzed for windows of various shapes. The ideal convolution path is a Hamiltonian path ending at the center of the window, the length of which is equal to the number of window coefficients. The simple architecture and control strategy make the proposed scheme suitable for VLSI implementation.

This paper presents a mapping strategy for parallel processing using an accurate characterization of the communication overhead. A set of objective functions is formulated to evaluate the optimality of mapping a problem graph onto a system graph. One of them is especially suitable for real-time applications of parallel processing. These objective functions are different from the conventional objective functions in that the edges in the problem graph are weighted and the actual distance rather than the nominal distance for the edges in the system graph is employed. This facilitates a more accurate quantification of the communication overhead. An efficient mapping scheme has been developed for the objective functions, where two levels of assignment optimization procedures are employed: initial assignment and pairwise exchange. The mapping scheme has been tested using the hypercube as a system graph.