Numerous studies have explored the neural correlates of musical rhythms using various neuroimaging modalities. Non-invasive neuroimaging modalities lack either the spatial or temporal resolution to reveal the nuances of neural processes involved in perception of musical rhythms. Intracranial recordings of electrophysiological activity such as electrocorticography (ECoG) can jointly provide spatial and temporal resolution for improved characterization and modeling of the underlying processes. The present study examines anticipatory and perceptual models that use ECoG recordings to estimate simple perceived and imagined musical rhythms in human participants. The resulting models are characterized and compared across participants. The results show that the anticipatory and perceptual models can reconstruct the auditory stimulus envelope with statistically-significant correlations when trained and tested on independent listening data. However, these models are unable to reliably reconstruct the expected rhythm pattern when trained on listening data and applied to imagining data. This suggests, similar to recent findings in overt and imagined speech decoding using intracranial signals, that there are likely distinct neural substrates activated during listening and imagining of musical rhythms.

Neurological disorders can severely impact speech communication. Recently, neural speech prostheses have been proposed that reconstruct intelligible speech from neural signals recorded superficially on the cortex. Thus far, it has been unclear whether similar reconstruction is feasible from deeper brain structures, and whether audible speech can be directly synthesized from these reconstructions with low-latency, as required for a practical speech neuroprosthetic. The present study aims to address both challenges. First, we implement a low-latency unit selection based synthesizer that converts neural signals into audible speech. Second, we evaluate our approach on open-loop recordings from 5 patients implanted with stereotactic depth electrodes who conducted a read-aloud task of Dutch utterances. We achieve correlation coefficients significantly higher than chance level of up to 0.6 and an average computational cost of 6.6 ms for each 10 ms frames. While the current reconstructed utterances are not intelligible, our results indicate promising decoding and run-time capabilities that are suitable for investigations of speech processes in closed-loop experiments.

Recent interest in virtual reality (VR) headsets has motivated research efforts to increase the user’s sense of immersion via feedback of physiological measures. This work presents the use of electroencephalographic (EEG) measurements during observation of immersive VR videos to estimate the user’s affective state. A pilot was conducted on 10 participants. Participants passively viewed a series of one-minute immersive VR video clips and subjectively rated the level of valence, arousal, liking, and dominance. Correlates between EEG spectral bands and the subjective ratings were analyzed to identify statistically significant frequencies and electrode locations across participants.

Recent studies have shown promise for designing Brain-Computer Interfaces (BCIs) to restore speech communication for those suffering from neurological injury or disease. Numerous BCIs have been developed to reconstruct different aspects of speech, such as phonemes and words, from brain activity. However, many challenges remain toward the successful reconstruction of continuous speech from brain activity during speech imagery. Here, we investigate the potential of differentiating speech and non-speech using intracranial brain activity in different frequency bands acquired by stereotactic EEG. The results reveal statistically significant information in the alpha and theta bands for detecting voice activity, and that using a combination of multiple frequency bands further improves performance with over 92% accuracy. Furthermore, the model is causal and can be implemented with low-latency for future closed-loop experiments. These preliminary findings show the potential of cross-frequency brain signal features for detecting speech activity to enhance speech decoding and synthesis models.

Direct synthesis from intracranial brain activity into acoustic speech might provide an intuitive and natural communication means for speech-impaired users. In previous studies we have used logarithmic Mel-scaled speech spectrograms (logMels) as an intermediate representation in the decoding from ElectroCorticoGraphic (ECoG) recordings to an audible waveform. Mel-scaled speech spectrograms have a long tradition in acoustic speech processing and speech synthesis applications. In the past, we relied on regression approaches to find a mapping from brain activity to logMel spectral coefficients, due to the continuous feature space. However, regression tasks are unbounded and thus neuronal fluctuations in brain activity may result in abnormally high amplitudes in a synthesized acoustic speech signal. To mitigate these issues, we propose two methods for quantization of power values to discretize the feature space of logarithmic Mel-scaled spectral coefficients by using the median and the logistic formula, respectively, to reduce the complexity and restricting the number of intervals. We evaluate the practicability in a proof-of-concept with one participant through a simple classification based on linear discriminant analysis and compare the resulting waveform with the original speech. Reconstructed spectrograms achieve Pearson correlation coefficients with a mean of r=0.5 +/- 0.11 in a 5-fold cross validation.

The integration of electroencephalogram (EEG) sensors into virtual reality (VR) headsets can provide the capability of tracking the user's cognitive state and eventually be used to increase the sense of immersion. Interactive VR applications that combine cognitive tasks and physical movements primarily introduce two types of contamination to the EEG signal: (1) those that result from the physical motion and (2) those that result from more subtle head and neck muscle tension. Each of these types of contamination can have distinct spatial and spectral characteristics, and thus should be processed differently. This study examines the suppression of both types of contamination in the EEG, and the respective impacts on cognitive workload classification.

The papers in this special section focus on biosignal-based spoken communication. Speech production is a complex process resulting from human activities initiated in the brain, eventually leading to muscle activities that produce respiratory, laryngeal, and articulatory gestures which finally create acoustic signals. Traditional speech processing systems capture and interpret the acoustic signal of speech. However, speech is not only limited to acoustics – speech-related activities can be measured at each level of speech processing, including the central and peripheral nervous systems, muscular action potentials, and speech kinematics. Their measurement, obtained through recordings from variety of sensor technologies, results in speech-related “biosignals” that have been studied for decades to better understand the underlying mechanisms of human speech processing. However, there is more: speech-related biosignals have the potential to overcome limitations of traditional acoustic-based systems for spoken communication. Biosignals can be captured before the airborne acoustic signal and are thus less prone to environmental noise. Also, they do not rely on the production of audible speech - both features open up newtracks for “Biosignalbased Spoken Communication”. Examples of these tracks include Brain-Computer Interfaces allowing for communication by directly decoding cortical brain activity into speech representations, and Silent-Speech Interfaces, which offer a way to communicate privately without disturbing bystanders and to restore spoken communication for people who lost their voice due to severe speech impairments. Furthermore, biosignals could provide valuable articulatory biofeedback to speakers about their own voice production for increasing articulatory awareness in speech therapy or language learning.

Speech is a complex process involving a wide range of biosignals, including but not limited to acoustics. These biosignals-stemming from the articulators, the articulator muscle activities, the neural pathways, and the brain itself-can be used to circumvent limitations of conventional speech processing in particular, and to gain insights into the process of speech production in general. Research on biosignal-based speech processing is a wide and very active field at the intersection of various disciplines, ranging from engineering, computer science, electronics and machine learning to medicine, neuroscience, physiology, and psychology. Consequently, a variety of methods and approaches have been used to investigate the common goal of creating biosignal-based speech processing devices for communication applications in everyday situations and for speech rehabilitation, as well as gaining a deeper understanding of spoken communication. This paper gives an overview of the various modalities, research approaches, and objectives for biosignal-based spoken communication.

Steady-state visual evoked potentials (SSVEP) are a class of signals obtained from the electroencephalogram (EEG) that are used in conjunction with brain-computer interfaces (BCIs). Inducing SSVEP signals requires flickering lights as stimuli, typically in the range of 5-45 Hz. However, due to low signal-to-noise ratio (SNR), SSVEP signals generated in certain frequency ranges can be difficult to detect. This paper studies cyclostationary-based detection for SSVEPs, which is a popular method for signal detection in low SNR environments, but whose application in the context of BCI systems has received only limited attention in the BCI research community. The results presented in the paper demonstrate that cyclostationary-based detection of SSVEP using spectral correlation density (SCD) performs as well as canonical correlation analysis (CCA), which is the most widely used method of SSVEP classification.

Steady-state visual evoked potentials (SSVEPs) are oscillations of the electroencephalogram (EEG) which are mainly observed over the occipital area that exhibits a frequency corresponding to a repetitively flashing visual stimulus. SSVEPs have proven to be very consistent and reliable signals for rapid EEG-based brain-computer interface (BCI) control. While a subject-specific SSVEP stimulus frequency optimization is ideal, this can be a tedious and time-consuming process. Thus, many studies select SSVEP stimulation frequencies somewhat arbitrarily. There is no standardized set of SSVEP stimulus frequencies or frequency selection method, and some studies even claim conflicting frequency ranges for optimal performance. In this work, 17 subjects were stimulated with an LED array that flashed according to a chirp-modulated signal having a frequency that varied linearly over the typical functional range of SSVEP. The resulting EEG was analyzed using canonical correlation analysis and a genetic algorithm was implemented to determine generalized stimulation frequency sets over a continuum of simulated multiclass BCI classification scenarios. The results show that distinct frequency feature groupings exist over the different multiclass scenarios, and that these groupings result in different information transfer rates. These offline results can provide a guide for generalized stimulus frequency selection for SSVEP-based BCIs with an arbitrary number of targets.

The goal of this study was to systematically compare signal characteristics and performance of three stimulation patterns that have been used to elicit steady-state somatosensory evoked potentials (SSSEPs): no pulses, random pulses, and rhythmic pulses with a consistent pattern. These three different vibrotactile stimulation patterns were provided to the fingertips of five healthy subjects by a small solenoid-type vibrating tactor. The five subjects, who were blindfolded, were asked to selectively focus their attention to either left or right fingertip flutter sensations, according to the audible cue given. Results of this study showed that small solenoid-type haptic tactors can elicit SSSEPs near the contra lateral central brain areas. There were significant differences in the resulting signals between the three paradigms, with the rhythmic pattern showing the highest classification accuracy. Moreover, the accuracy of the rhythmic pulse pattern was significantly higher than with or without random pulse patterns for the majority of subjects. The results of this study should provide insights to future research of SSSEP based BCIs and hybrid BCIs that use SSSEPS as one type of brain signal for users who are unable to use visual BCIs.

A hierarchical elastic computer-aided detection algorithm is proposed to automatically detect the horizon in an aerial image. A hierarchical strategy, including coarse-level detection and fine-level adjustment, is applied. First, the original image is blurred by a large-scale low-pass filter. Then, a Canny edge detector and Hough transform are successively utilized to find major edges in the image and identify lines associated with those major edges. The desired horizon is modeled by the resulting line that best satisfies certain criteria. By doing so, the general position of the horizon can be quickly detected at the coarse-level step. Since the horizon is often not a straight line, an elastic fine-level adjustment is applied to capture the precise curvature of the horizon. A quantitative performance metric is designed, and preliminary experimental results show the feasibility and reliability of the proposed algorithm.

Recent studies have demonstrated that the P300 Speller can be used to reliably control a brain-computer interface via stereotactic depth electrodes (SDEs) implanted in and adjacent to the hippocampus. The superior bandwidth and spatial resolution of intracranial signals compared to scalp electroencephalography (EEG) may provide better insight into the nature of the responses evoked by the P300 Speller. Furthermore, the signals acquired by SDEs provide a new glimpse at activity from deeper brain structures, with respect to well-established scalp-EEG characterization of the P300 Speller responses. This study examines the spatio-temporal progression of the P300 Speller response from SDEs in and adjacent to the hippocampus for six conventional frequency bands.

Recently particle swarm optimization (PSO) has been studied for use in adaptive filtering problems where the mean squared error (MSE) surface is ill-conditioned. Although the swarm generally converges to a limit point, when the population of the swarm is small the entire swarm often stagnates before reaching the global minimum on the MSE surface. This paper examines enhancements designed to improve the performance of the conventional PSO algorithm. It is shown that an enhanced PSO algorithm, called the Modified PSO (MPSO) algorithm, is quite effective in achieving global convergence for IIR and nonlinear adaptive filters

The paper investigates the application of a modified particle swarm optimization technique to nonparametric density estimation based independent component analysis (ICA). Nonparametric ICA has the advantage over traditional ICA techniques in that its performance is not dependent upon prior assumptions about the source distributions. Particle swarm optimization (PSO) is similar to the genetic algorithm in that it utilizes a population based search suitable for optimizing multimodal error surfaces where gradient-based algorithms tend to fail, such as those generated by nonlinear entropy maximization schemes used in ICA algorithms.

This paper introduces the application of particle swarm optimization techniques to infinite impulse response (IIR) adaptive filter structures. Particle swarm optimization (PSO) is similar to the genetic algorithm (GA) in that it performs a structured randomized search of an unknown parameter space by manipulating a population of parameter estimates to converge on a suitable solution. Unlike the genetic algorithm, particle swarm optimization has not emerged in adaptive filtering literature. Both techniques are independent of the adaptive filter structure and are capable of converging on the global solution for multimodal optimization problems, which makes them especially useful for optimizing IIR and nonlinear adaptive filters. This paper outlines PSO and provides a comparison to the GA for IIR filter structures.

This paper investigates the application of particle swarm optimization techniques to infinite impulse response (IIR) adaptive phase equalizers. Particle swarm optimization (PSO) is similar to the genetic algorithm (GA) in that it utilizes a population based search suitable for optimizing multimodal error surfaces where gradient-based algorithms tend to fail, such as those generated by IIR adaptive filters. This paper investigates PSO for the phase equalization of minimum phase surface acoustic wave (SAW) filters used in CDMA receivers.