Drowsiness significantly contributes to accidents and decreased performance due to its impact on attention and concentration. Various sleepiness detection methods have been proposed but have not been widely adopted in daily life due to several limitations: (1) the lack of long-term measurement capability, (2) the need for external assistance for the operation, and (3) without the capability of reliable detection from drowsiness to sleep onset. In this pilot study, we investigated the characteristics of the forehead EEG from alertness sleep. The two-phase experiment was designed. In phase 1, subjects (N=13) had to sit in a chair and play a Sudoku puzzle, representing cognitive engagement in an altered state. In Phase 2, they lay on a bed and watched a first-person-perspective driving video record, which involved transitions from alertness to drowsiness and drowsiness to sleep. Validated with video data, forehead EEG detected an increase in eye-blink frequency, and the width of eye-blink peaks was also increased. After subjects fell asleep, forehead EEG exhibited alpha rhythms and slow eye movements, the mixing patterns observed in EEG and EOG. similar to standard EEG and EOG recordings. Moreover, our participants demonstrated a shorter sleep onset latency (1.23 ± 0.83 min) than previous sleep studies without fatigue induction. By integrating these characteristics with EEG- and EOG-based sleep score methods, forehead EEG presents a practical solution as a wearable device for daily drowsiness detection.

Accurate localization of the vocal folds is critical for diagnostic and therapeutic applications in medical imaging. This paper developed a deep learning-based object detection system to address different tasks related to vocal fold detection. A two-step transfer learning approach was proposed for model training. The YOLOv8 was pre-trained with a large scale of high-speed video recordings available in a public dataset. Then, we fine-tuned the model using a 1:1 combination of the data from the public dataset and the low-frame rate (30 frames/sec) dataset collected from the hospital CGMH in Taiwan to fine-tune the final optimized glottis detection model. The results show that the recall and precision of ROI multiple bounding box predictions are 97.6% and 98.3%, respectively. In comparison, the recall and precision of single bounding box predictions are 97.5% and 100%, respectively. These results demonstrate the successful use of deep learning technology for vocal fold localization with superior performance. Our study provides valuable information for selecting appropriate object detection modalities in medical imaging applications for diagnosis and treatment planning in laryngology and otorhinolaryngology.

This paper explores silent speech decoding in active brain-computer interface (BCI) systems, which offer more natural and flexible communication than traditional BCI applications. We collected a new silent speech dataset of over 120 hours of electroencephalogram (EEG) recordings from 12 subjects, capturing 24 commonly used English words for language model pretraining and decoding. Following the recent success of pretraining large models with self-supervised paradigms to enhance EEG classification performance, we propose Large Brain Language Model (LBLM) pretrained to decode silent speech for active BCI. To pretrain LBLM, we propose Future Spectro-Temporal Prediction (FSTP) pretraining paradigm to learn effective representations from unlabeled EEG data. Unlike existing EEG pretraining methods that mainly follow a masked-reconstruction paradigm, our proposed FSTP method employs autoregressive modeling in temporal and frequency domains to capture both temporal and spectral dependencies from EEG signals. After pretraining, we finetune our LBLM on downstream tasks, including word-level and semantic-level classification. Extensive experiments demonstrate significant performance gains of the LBLM over fully-supervised and pretrained baseline models. For instance, in the difficult cross-session setting, our model achieves 47.2% accuracy on semantic-level classification and 42.3% in word-level classification, outperforming baseline methods substantially. Our research advances silent speech decoding in active BCI systems, offering an innovative solution for EEG language model pretraining and a new dataset for fundamental research.

This paper presents MindSpeak, a real-time brain-computer interface (BCI) system for recording, processing, and decoding silent speech to enable online multimodal communication between the human brain and a computer, involving both noninvasive multichannel EEG signals and text output. To enable hand-free and brain-only control, our system incorporates steady-state visual evoked potential (SSVEP) for users to select incomplete sentences from a predefined pool and confirm the correctness of decoded words. An intuitive graphical interface is designed for natural communication. We evaluate the effectiveness of our real-time BCI system, which achieves 77.3% accuracy in decoding silent speech and 98.9% accuracy in SSVEP-based selection and confirmation of correct sentences. Unlike existing BCI systems, the presented MindSpeak system significantly expands the application scope of existing BCI systems by enabling users to express complete thoughts through a fully BCI-controlled interactive interface. Our demonstration video is on: https://youtu.be/B1wt1dmCCrg.

Electroencephalography (EEG) plays a crucial role in the diagnosis of various neurological disorders. However, small hospitals and clinics often lack advanced EEG signal analysis systems and are prone to misinterpretation in manual EEG reading. This study proposes an innovative hybrid artificial intelligence (AI) system for automatic interpretation of EEG background activity and report generation. The system combines deep learning models for posterior dominant rhythm (PDR) prediction, unsupervised artifact removal, and expert-designed algorithms for abnormality detection. For PDR prediction, 1530 labeled EEGs were used, and the best ensemble model achieved a mean absolute error (MAE) of 0.237, a root mean square error (RMSE) of 0.359, an accuracy of 91.8% within a 0.6 Hz error, and an accuracy of 99% within a 1.2 Hz error. The AI system significantly outperformed neurologists in detecting generalized background slowing (p = 0.02; F1: AI 0.93, neurologists 0.82) and demonstrated improved focal abnormality detection, although not statistically significant (p = 0.79; F1: AI 0.71, neurologists 0.55). Validation on both an internal dataset and the Temple University Abnormal EEG Corpus showed consistent performance (F1: 0.884 and 0.835, respectively; p = 0.66), demonstrating generalizability. The use of large language models (LLMs) for report generation demonstrated 100% accuracy, verified by three other independent LLMs. This hybrid AI system provides an easily scalable and accurate solution for EEG interpretation in resource-limited settings, assisting neurologists in improving diagnostic accuracy and reducing misdiagnosis rates.

Virtual reality (VR), a computer-generated interactive environment, is provided to a user by projecting a peripheral image onto environmental surfaces. VR has an advantage of enhancing the immersive experience. Nowadays, VR has been widely applied in tourism and cultural experience. On the other hand, a recent integration of electroencephalography-based (EEG-based) brain-computer interface (BCI) and VR is capable of promoting the immersive virtual experience. Therefore, our study aims to propose an integrative framework to implement EEG-based BCI in a VR game to advance the cultural experience. A room escape game in a Tainan temple is created. EEG signals arc recorded while users arc playing the game. The online analyses of EEG signals arc used to interact with the VR display. This integrative framework can result in a better experience than the conventional setup.

This paper presents the closed-loop neural disorder control concept and some design considerations. Two architectures of closed-loop neuromodulation for Parkinson's disease and epileptic seizure are proposed. One is a closed-loop deep brain stimulator, which meets the IEC 60601-1 standard. The other one is an implantable SoC for epileptic seizure control, which is verified by animal experiment.

Atrial fibrillation (AF) is the most frequent cardiac arrhythmia seen in clinical practice. Several therapeutical approaches have been developed to terminate the AF and the effects are evaluated by the reduction of the wavelet number after the treatments. Most of the previous studies focus on modeling and analysis the mechanism, and the characteristic of AF. But no one discusses about the prediction of the result after the drug treatment. This paper is the first study to predict whether the drug treatment for AF is active or not. In this paper, the linear autoregressive model with exogenous inputs (ARX) that models the system output-input relationship by solving linear regression equations with least squares method was developed and applied to estimate the effects of pharmacological therapy on AF. Recordings (224-site bipolar recordings) of plaque electrode arrays placed on the right and left atria of pigs with sustained AF induced by rapid atrial-pacing were used to train and test the ARX models. The cardiac mapping data from twelve pigs treated with intravenous administration of antiarrhythmia drug, propafenone (PPF) or dl-sotalol (STL), was evaluated. The recordings of cardiac activity before the drug treatment were input to the model and the model output reported the estimated wavelet number of atria after the drug treatment. The results show that the predicting accuracy rate corresponding to the PPF and STL treatment was 100% and 92%, respectively. It is expected that the developed ARX model can be further extended to assist the clinical staffs to choose the effective treatments for the AF patients in the future.

In this paper, a fuzzy inference system for sleep staging was developed. Nine input variables including temporal and spectrum analyses of the EEG, EOG, and EMG signals were extracted and normalization was applied to these variables to reduce the effect of individual variability. A fuzzy inference system contains fourteen fuzzy rules was designed to classify the 30-s sleep epochs as five sleep stages. Finally, a smoothing process was applied to the scoring results for fine-tuning. The average accuracy of the proposed method applied to 16 all-night polysomnography (PSG) recordings compared with the manual scorings can reach 87 %. This method can integrate with various PSG systems for sleep monitoring in clinical or homecare applications.

This paper applies fuzzy vector quantization (FVQ) to the modeling of observation-based Discrete Hidden Markov Model (DHMM) and then to improve the speech recognition rate for the Mandarin speech. Vector quantization based on a codebook is a fundamental process to recognize the speech signal by DHMM. A codebook will be first trained by K means algorithms using Mandarin training speech. Then, based on the trained codebook, the speech features are quantized by the fuzzy sets defined on each vectors of the codebook. Subsequently, the quantized speech features are statistically applied to train the model of DHMM for the speech recognition. All the speech features to be recognized should go through the FVQ based on the fuzzy codebook before being fed into the DHMM model for recognition. Experimental results in this paper shows that the speech recognition rate can be improved by using FVQ algorithm to train the model of DHMM.

Approximately 1% of people in the world have epilepsy and 25% of epilepsy patients cannot be treated sufficiently by any available therapy. An automatic seizure detection system can reduce the time taken to review the EEG data by the neurologist for epilepsy diagnosis. In this paper, various EEG features integrated with the linear or non-linear classifiers are evaluated for seizure detection. For the EEG features, approximate entropy (ApEn) combined with 1) EEG power spectra or 2) autoregressive model (AR) are compared. In addition, the principle component analysis (PCA) is also utilized for feature extraction. For the classifiers, two linear models, linear least square (LLS) and linear discriminant analysis (LDA), and two nonlinear models, backpropagation neural network (BPNN) and support vector machine with radial basis function kernel (RBFSVM) are compared. The EEG signals of three Long Evans rats with spontaneous absence seizures are used for leave-one-out cross-validation. Experimental results shows that combining ApEn and multi-band EEG power spectra are superior to the combination of ApEn and AR model for all classifiers. The best average accuracy is 97.5% performed by RBFSVM and the linear models can achieve to higher than 95%. The automatic seizure detection method can be utilized to drive the seizure warning device or seizure control devices in the future to enhance the patients' quality of life.

Music provides a tool to study numerous aspects of neuroscience from motion-skill to emotion since listening to and producing music involves many brain functions. The musician's brain is also regarded as an ideal model to investigate plasticity of the human brain. In this paper, an EEG-based neural network is proposed to assess neuroplasticity induced by musical training. A musical chord perception experiment is designed to acquire and compare the behavioral and neural responses of musicians and non-musicians. The ERPs elicited by the consonant and dissonant chords are combined together as the features of the model. The principle component analysis (PCA) is used to reduce feature dimensions and the dimension-reduced features are input to a feedforward neural network to recognize the brain potentials belong to a musician or a non-musician. The accuracy can reach 97% in average for leave-one-out cross validation of six subjects in this experiment. It demonstrates the feasibility of assessing effects of musical training by ERP signals elicited by musical chord perception.

Monitoring and predicting human cognitive state and performance using physiological signals such as Electroencephalogram (EEG) have recently gained increasing attention in the fields of brain-computer interface and cognitive neuroscience. Most previous psychophysiological studies of cognitive changes have attempted to use the same model for all subjects. However, the relatively large individual variability in EEG dynamics relating to loss of alertness suggests that for many operators, group statistics cannot be used to accurately predict changes in cognitive states. Attempts have also been made to build a subject-dependent model for each individual based on his/her pilot data to account for individual variability. However, such methods assume the cross-session variability in EEG dynamics to be negligible, which could be problematic due to electrode displacements, environmental noises, and skin-electrode impedance. Here first we show that the EEG power in the alpha and theta bands are strongly correlated with changes in the subject’s cognitive state reflected through his driving performance and hence his departure from alertness. Then under very mild and realistic assumptions we derive a model for the alert state of the person using EEG power in the alpha and theta bands. We demonstrate that deviations (computed by Mahalanobis distance) of the EEG power in the alpha and theta bands from the corresponding alert models are correlated to the changes in the driving performance. Finally, for detection of drowsiness we use a linear combination of deviations of the EEG power in the alpha band and theta band from the respective alert models that best correlates with subject’s changing level of alertness, indexed by subject’s behavioral response in the driving task. This approach could lead to a practical system for noninvasive monitoring of the cognitive state of human operators in attention-critical settings.

Driver distraction has been recognized as a significant cause of traffic incidents. Therefore, the aim of this study was to investigate electroencephalography (EEG) dynamics in response to distraction during driving. To study human cognition under specific driving task, we used virtual reality (VR) based driving simulation to simulate events including unexpected car deviations and mathematics questions (math) in real driving. For further assessing effects of the stimulus onset asynchrony (SOA) between the deviation onset and math presented on the EEG dynamics, we designed five cases with different SOA. The scalp-recorded EEG channel signals were first separated into independent brain sources by independent component analysis (ICA). Then, the event-related-spectral-perturbations (ERSP) measuring changes of EEG power spectra were used to evaluate the brain dynamics in time-frequency domains. Results showed that increases of theta band (5~7.8 Hz) and beta band (12.2~17 Hz) power were observed in the frontal cortex. Results demonstrated that reaction time and multiple cortical EEG sources responded to the driving deviations and math occurrences differentially in the stimulus onset asynchrony. Results also suggested that the theta band power increase in frontal area could be used as the distracted indexes for early detecting driver's inattention in the future.

Neural prosthetic technologies have helped many patients by restoring vision, hearing, or movement and relieving chronic pain or neurological disorders. While most neural prosthetic systems to date have used invasive or implantable devices for patients with inoperative or malfunctioning external body parts or internal organs, a much larger population of “healthy” people who suffer episodic or progressive cognitive impairments in daily life can benefit from noninvasive neural prostheses. For example, reduced alertness, lack of attention, or poor decision-making during monotonous, routine tasks can have catastrophic consequences. This study proposes a noninvasive mobile prosthetic platform for continuously monitoring high-temporal resolution brain dynamics without requiring application of conductive gels on the scalp. The proposed system features dry microelectromechanical system electroencephalography sensors, low-power signal acquisition, amplification and digitization, wireless telemetry, online artifact cancellation, and signal processing. Its implications for neural prostheses are examined in two sample studies: 1) cognitive-state monitoring of participants performing realistic driving tasks in the virtual-reality-based dynamic driving simulator and 2) the neural correlates of motion sickness in driving. The experimental results of these studies provide new insights into the understanding of complex brain functions of participants actively performing ordinary tasks in natural body positions and situations within real operational environments.

In this paper, a novel two-stage noise removal algorithm to deal with impulse noise is proposed. In the first stage, an adaptive two-level feedforward neural network (NN) with a backpropagation training algorithm was applied to remove the noise cleanly and keep the uncorrupted information well. In the second stage, the fuzzy decision rules inspired by the human visual system (HVS) are proposed to classify the image pixels into human perception sensitive class and nonsensitive class, and to compensate the blur of the edge and the destruction caused by the median filter. An NN is proposed to enhance the sensitive regions with higher visual quality. According to the experimental results, the proposed method is superior to conventional methods in perceptual image quality as well as the clarity and smoothness in edge regions.

Accidents caused by errors and failures in human performance among traffic fatalities have a high rate causing death and become an important issue in public security. The key problem causing these car accidents is mainly because that the drivers failed to perceive the changes of the traffic lights or the unexpected conditions happening accidentally on the roads. In this paper, we devised a quantitative analysis for ongoing assessment of driver's cognitive responses by investigating the neurobiological information underlying electroencephalographic (EEG) brain dynamics in traffic-light experiments in a virtual-reality (VR) dynamic driving environment. Three different feature extraction methods including nonparametric weighted feature extraction (NWFE), principal component analysis (PCA), discriminant analysis feature extraction (DAFE) are applied to reduce the feature dimension and project the measured EEG signals to a feature space spanned by their eigenvectors. After that, the mapped data can be classified with fewer features and their classification results are compared by utilizing three different classifiers including Gaussian classifier (GC), k nearest neighbor classification (KNNC), and naive Bayes classifier (NBC). Experimental results show that the successful rate of nonparametric weighted feature extraction combined with Gaussian classifier is higher more than 10% compared with other combinations. It also demonstrates the feasibility of detecting and analyzing single-trail ERP signals that represent operators' cognitive states and responses to task events.

In this paper, a novel human visual system (HVS)-directed neural-network-based adaptive interpolation scheme for natural image is proposed. A fuzzy decision system built from the characteristics of the HVS is proposed to classify pixels of the input image into human perception nonsensitive class and sensitive class. Bilinear interpolation is used to interpolate the nonsensitive regions and a neural network is proposed to interpolate the sensitive regions along edge directions. High-resolution digital images along with supervised learning algorithms are used to automatically train the proposed neural network. Simulation results demonstrate that the proposed new resolution enhancement algorithm can produce a higher visual quality for the interpolated image than the conventional interpolation methods.

Machine vision is a key technology used in an intelligent transportation system (ITS) to augment human drivers' visual capabilities. For the in-car applications, additional motion components are usually induced by disturbances such as the bumpy ride of the vehicle or the steering effect, and they will affect the image interpretation processes that is required by the motion field (motion vector) detection in the image. In this paper, a novel robust in-car digital image stabilization (DIS) technique is proposed to stably remove the unwanted shaking phenomena in the image sequences captured by in-car video cameras without the influence caused by moving object (front vehicles) in the image or intentional motion of the car, etc. In the motion estimation process, the representative point matching (RPM) module combined with the inverse triangle method is used to determine and extract reliable motion vectors in plain images that lack features or contain a large low-contrast area to increase the robustness in different imaging conditions, since most of the images captured by in-car video cameras include large low-contrast sky areas. An adaptive background evaluation model is developed to deal with irregular images that contain large moving objects (front vehicles) or a low-contrast area above the skyline. In the motion compensation processing, a compensating motion vector (CMV) estimation method with an inner feedback-loop integrator is proposed to stably remove the unwanted shaking phenomena in the images without losing the effective area of the images with a constant motion condition. The proposed DIS technique was applied to the on-road captured video sequences with various irregular conditions for performance demonstrations

The public security has become an important issue in recent years, especially, the safe manipulation and control of vehicles in preventing the growing number of traffic accident fatalities. Accidents caused by drivers' drowsiness have a high fatality rate due to the decline of drivers' abilities in perception, recognition, and vehicle control abilities while sleepy. Preventing such an accident requires a technique for detecting, estimating, and predicting the level of alertness of a driver and a mechanism to maintain the driver's maximum performance of driving. The ICAFNN is a fuzzy neural network (FNN) capable of parameter self-adapting and structure self-constructing to acquire a small number of fuzzy rules for interpreting the embedded knowledge of a system from the given training data set. Our experiments show that the ICAFNN can achieve significant improvements in the accuracy of drowsiness estimation compared with our previous works.

In this paper, a portable real-time wireless embedded brain signal acquisition/processing system is developed. The proposed system integrates electroencephalogram signal amplifier technique, wireless transmission technique, and embedded real-time system. The development strategy of this system contains three parts: First, the Bluetooth protocol is used as a transmission interface and integrated with the bio-signal amplifier to transmit the measured physiological signals wirelessly. Second, the OMAP (open multimedia architecture platform) is used as a development platform and an embedded operating system for OMAP is also designed. Finally, DSP Gateway is developed as a mechanism to deal with the brain-signal analyzing tasks shared by ARM and DSP. A driver's cognitive-state estimation program has been developed and implemented on the proposed dual core processor-based real time wireless embedded system for demonstration.

Driving safely has received increasing attention of the publics due to the growing number of traffic accidents that the driver's driving style is highly correlated to many accidents. The purpose of this study is to investigate the relationship between driver's driving style and driver's ERP response. In our research, a virtual reality (VR) driving environment is developed to provide stimuli to subjects. Independent component analysis (ICA) is used to decompose the electroencephalogram (EEG) data. The power spectrum analysis of ICA components and correlation analysis are employed to investigate the EEG activities related to driving style. Experimental results demonstrate that we may classify the drivers into aggressive or gentle styles based on the observed ERP difference corresponding to the proposed unexpected obstacle dodging tasks.

Fuzzy neural networks (FNNs) for pattern classification usually use the backpropagation or C-cluster type learning algorithms to learn the parameters of the fuzzy rules and membership functions from the training data. However, such kinds of learning algorithms usually cannot minimize the empirical risk (training error) and expected risk (testing error) simultaneously, and thus cannot reach a good classification performance in the testing phase. To tackle this drawback, a support-vector-based fuzzy neural network (SVFNN) is proposed for pattern classification in this paper. The SVFNN combines the superior classification power of support vector machine (SVM) in high dimensional data spaces and the efficient human-like reasoning of FNN in handling uncertainty information. A learning algorithm consisting of three learning phases is developed to construct the SVFNN and train its parameters. In the first phase, the fuzzy rules and membership functions are automatically determined by the clustering principle. In the second phase, the parameters of FNN are calculated by the SVM with the proposed adaptive fuzzy kernel function. In the third phase, the relevant fuzzy rules are selected by the proposed reducing fuzzy rule method. To investigate the effectiveness of the proposed SVFNN classification, it is applied to the Iris, Vehicle, Dna, Satimage, Ijcnn1 datasets from the UCI Repository, Statlog collection and IJCNN challenge 2001, respectively. Experimental results show that the proposed SVFNN for pattern classification can achieve good classification performance with drastically reduced number of fuzzy kernel functions.

During the past years, the public security has become an important issue, especially, the safe manipulation and control of various vehicles. Maintaining high cognition is particularly important for the drivers behind the steering wheel. It requires an optimal estimation system to online continuously detect drivers' cognitive state related to abilities in perception, recognition, and vehicle control. In this paper, we proposed an EEG-based alertness estimation system with automatic feature selection mechanism. The independent component analysis (ICA) is used first to decompose the measured electroencephalogram (EEG). Then, a time-frequency analysis is performed to evaluate the time-frequency characteristic of each ICA component. We also proposed a new adaptive feature extracting mechanism for selections of frequency bands and ICA components. Different ranges of the alpha rhythm of subjects can be evaluated by the adaptive feature extracting mechanism according to the correlation coefficient between the ICA time-frequency response and the driving performance. The extracted features are then trained both by linear regression model and self-constructing neuro-fuzzy inference network (SONFIN) for the estimation of driving performance. The training and testing results of SONFIN are 96% and 91%, while the results of linear regression model are 90% and 85%, respectively. It demonstrates that the proposed adaptive feature extracting mechanism can achieve a great performance in alertness estimation with frequency band and component selection.

A fuzzy neural network based on support vector learning mechanism for function approximation is proposed in this paper. Support vector regression (SVR) is a novel method for tackling the problems of function approximation and regression estimation based on the statistical learning theory. SVR has been shown to have robust properties against noise. A novel support-vector-regression based fuzzy neural network (SVRFNN) by integrating SVR technology into FNN is developed. The SVRFNN combines the high accuracy and robustness of support vector regression (SVR) and the efficient human-like reasoning of FNN for function approximation. Experimental results show that the proposed SVFNN for function approximation can achieve good approximation performance with drastically reduced number of fuzzy kernel functions.

This paper proposes a novel neural-network-based adaptive hybrid-reflectance three-dimensional (3-D) surface reconstruction model. The neural network automatically combines the diffuse and specular components into a hybrid model. The proposed model considers the characteristics of each point and the variant albedo to prevent the reconstructed surface from being distorted. The neural network inputs are the pixel values of the two-dimensional images to be reconstructed. The normal vectors of the surface can then be obtained from the output of the neural network after supervised learning, where the illuminant direction does not have to be known in advance. Finally, the obtained normal vectors are applied to enforce integrability when reconstructing 3-D objects. Facial images and images of other general objects were used to test the proposed approach. The experimental results demonstrate that the proposed neural-network-based adaptive hybrid-reflectance model can be successfully applied to objects generally, and perform 3-D surface reconstruction better than some existing approaches.

Photometric stereo technique deals with the reconstruction of three-dimensional (3-D) shape of an object by using several images of the same surface taken from the same viewpoint but under illuminations from different directions. In this paper, we propose a new photometric stereo scheme based on a new reflectance model and the postnonlinear (PNL) independent components analysis (ICA) method. The proposed nonlinear reflectance model consists of diffuse components and specular components for modeling the surface reflectance of a stereo object in an image. Unlike the previous approaches, these two components are not separated and processed individually in the proposed model. An unsupervised learning adaptation algorithm is developed to estimate the reflectance model based on image intensities. In this algorithm, the PNL ICA method is used to obtain the surface normal on each point of an image. Then, the 3-D surface model is reconstructed based on the estimated surface normal on each point of image by using the enforcing integrability method. Two experiments are performed to assess the performance of the proposed approach. We test our algorithm on synthetically generated images for the reconstruction of surface of objects and on a number of real images captured from the Yale Face Database B. These testing images contain variability due to illumination and varying albedo in each point of surface of human faces. All the experimental results are compared to those of the existing photometric stereo approaches tested on the same images. The results clearly indicate the superiority of the proposed nonlinear reflectance model over the conventional Lambertian model and the other linear hybrid reflectance model.

The growing number of traffic fatalities in recent years has become a serious concern to society. Accidents caused by drivers' drowsiness behind the steering wheel have a high fatality rate because of the marked decline in the drivers' abilities of perception, recognition, and vehicle control abilities while sleepy. Preventing accidents caused by drowsiness requires a technique for detecting, estimating, and predicting the level of alertness of a driver and a mechanism for maintaining his/her maximum performance. This work describes a system that combines electroencephalographic (EEG) power spectrum estimation, principal component analysis, and fuzzy neural network model to estimate/predict drivers' drowsiness level in a driving simulator. Our results demonstrated that, for the first time, it is feasible to accurately estimate task performance, accurately estimate quantitatively measured driving performance, expressed as deviation between the center of the vehicle and the center of the cruising lane, in a realistic driving simulation.

In this paper, a new technique for the Chinese text-to-speech (TTS) system is proposed. Our major effort focuses on the prosodic information generation. New methodologies for constructing fuzzy rules in a prosodic model simulating human's pronouncing rules are developed. The proposed Recurrent Fuzzy Neural Network (RFNN) is a multilayer recurrent neural network (RNN) which integrates a Self-cOnstructing Neural Fuzzy Inference Network (SONFIN) into a recurrent connectionist structure. The RFNN can be functionally divided into two parts. The first part adopts the SONFIN as a prosodic model to explore the relationship between high-level linguistic features and prosodic information based on fuzzy inference rules. As compared to conventional neural networks, the SONFIN can always construct itself with an economic network size in high learning speed. The second part employs a five-layer network to generate all prosodic parameters by directly using the prosodic fuzzy rules inferred from the first part as well as other important features of syllables. The TTS system combined with the proposed method can behave not only sandhi rules but also the other prosodic phenomena existing in the traditional TTS systems. Moreover, the proposed scheme can even find out some new rules about prosodic phrase structure. The performance of the proposed RFNN-based prosodic model is verified by imbedding it into a Chinese TTS system with a Chinese monosyllable database based on the time-domain pitch synchronous overlap add (TD-PSOLA) method. Our experimental results show that the proposed RFNN can generate proper prosodic parameters including pitch means, pitch shapes, maximum energy levels, syllable duration, and pause duration. Some synthetic sounds are online available for demonstration.

In this paper, a novel HVS-directed neural-network-based adaptive interpolation scheme for natural image is proposed. A fuzzy decision system built from the characteristics of the human visual system (HVS) is proposed to classify pixels of the input image into human perception non-sensitive class and sensitive class. High-resolution digital images along with supervised learning algorithms are used to automatically train the proposed neural network. Simulation results demonstrate that the proposed new resolution enhancement algorithm can produce higher visual quality of the interpolated image than the conventional interpolation methods.

A new approach is proposed that closely synthesizes tones of plucked string instruments by using a class of physical modeling recurrent networks. The strategies employed consist of a fast training algorithm and a multistage training procedure that are able to obtain the synthesis parameters for a specific instrument automatically. The training vector can be recorded tones of most target plucked instruments with ordinary microphones. The proposed approach delivers encouraging results when it is applied to different types of plucked string instruments such as steel-string guitar, nylon-string guitar, harp, Chin, Yueh-chin, and Pipa. The synthesized tones sound very close to the originals produced by their acoustic counterparts. In addition, the paper presents an embedded technique that can produce special effects such as vibrato and portamento that are vital to the playing of plucked-string instruments. The computation required in the resynthesis processing is also reasonable.

In this paper, an automatic way of analyzing/synthesizing plucked-string instruments by using a simple IIR (infinite impulse response) structure is presented. The basic structure of the proposed IIR filter consists of a delay line with feedback loops pulled out from the delay line back to the input side of the filter. The feedback portion acts like a prediction filter to modify the signal for the next period such that it can be as close to the original tone as possible. The delay line and the prediction filter provide the mechanism to produce the desired pitch of the synthetic sound. The design of the coefficients for this IIR synthesizer is accomplished by using a neural network based training algorithm. The excitation signal is extracted from the original tones recorded from the target instrument. Several sets of filter coefficients are required for different synthesis stages such as the attacking part and sustain part. The proposed algorithm is tested over a wide range of plucked-string instruments. Although it requires more computation compared to some model-based approaches, the analysis work is simple and straightforward. In addition, the synthetic tone sounds very close to the correspondent original sound.

A physical modeling method for electronic music synthesis of plucked-string tones by using recurrent networks is proposed. A scattering recurrent network (SRN) which is used to analyze string dynamics is built based on the physics of acoustic strings. The measured vibration of a plucked string is employed as the training data for the supervised learning of the SRN. After the network is well trained, it can be regarded as the virtual model for the measured string and used to generate tones which can be very close to those generated by its acoustic counterpart. The "virtual string" corresponding to the SRN can respond to different "plucks" just like a real string, which is impossible using traditional synthesis techniques such as frequency modulation and wavetable. The simulation of modeling a cello "A"-string demonstrates some encouraging results of the new music synthesis technique. Some aspects of modeling and synthesis procedures are also discussed.

A new recurrent-network-based synthesis technique for general plucked-string instruments is proposed. If is expected that the proposed technique can serve as a synthesis model as well as a systematic parameter-searching engine such that the synthetic tones can sound as close to the recorded tone as possible. Two traditional Chinese plucked-string instruments, Pipa and Qin, are introduced. A multi-stage training procedure and an improved training algorithm for the determination of synthesis parameters and excitation signal are presented. The analysis/synthesis results of the two Chinese traditional plucked-string instruments considered are shown.