The human brain tracks amplitude fluctuations of both speech and music, which reflects acoustic processing in addition to the encoding of higher-order features and one's cognitive state. Comparing neural tracking of speech and music envelopes can elucidate stimulus-general mechanisms, but direct comparisons are confounded by differences in their envelope spectra. Here, we use a novel method of frequency-constrained reconstruction of stimulus envelopes using EEG recorded during passive listening. We expected to see music reconstruction match speech in a narrow range of frequencies, but instead we found that speech was reconstructed better than music for all frequencies we examined. Additionally, models trained on all stimulus types performed as well or better than the stimulus-specific models at higher modulation frequencies, suggesting a common neural mechanism for tracking speech and music. However, speech envelope tracking at low frequencies, below 1 Hz, was associated with increased weighting over parietal channels, which was not present for the other stimuli. Our results highlight the importance of low-frequency speech tracking and suggest an origin from speech-specific processing in the brain.

Freezing of gait (FOG), an episodic gait disturbance characterized by the inability to generate effective stepping, occurs in more than half of Parkinson's disease patients. It is associated with both executive dysfunction and attention and becomes most evident during dual tasking (performing two tasks simultaneously). This study examined the effect of dual motor-cognitive virtual reality training on dual-task performance in FOG. Twenty community dwelling participants with Parkinson's disease (13 with FOG, 7 without FOG) participated in a pre-assessment, eight 20-minute intervention sessions, and a post-assessment. The intervention consisted of a virtual reality maze (DFKI, Germany) through which participants navigated by stepping-in-place on a balance board (Nintendo, Japan) under time pressure. This was combined with a cognitive task (Stroop test), which repeatedly divided participants' attention. The primary outcome measures were pre- and post-intervention differences in motor (stepping time, symmetry, rhythmicity) and cognitive (accuracy, reaction time) performance during single- and dual-tasks. Both assessments consisted of 1) a single cognitive task 2) a single motor task, and 3) a dual motor-cognitive task. Following the intervention, there was significant improvement in dual-task cognitive and motor parameters (stepping time and rhythmicity), dual-task effect for those with FOG and a noteworthy improvement in FOG episodes. These improvements were less significant for those without FOG. This is the first study to show benefit of a dual motor-cognitive approach on dual-task performance in FOG. Advances in such virtual reality interventions for home use could substantially improve the quality of life for patients who experience FOG.

Abstract This study investigates the use of speech as an indicator of the onset of cognitive decline in the elderly. The analysis found features that correlate with the results of a clinical measure of cognitive function. Using a combination of temporal language features, such as pause and utterance duration, 76% classification accuracy was achieved. While no significant results were found for ASR experiments, vowel duration, on average, increased by 17% for subjects with cognitive impairment compared to those without. The results from this study introduce the concept of longitudinal studies into aging using speech as a window into cognitive function. Index Terms: cognitive decline; elderly, temporal features 1. Introduction The world’s population is growing older. In the next 50 years, the number of older people is forecast to quadruple, growing from about 600 million to almost 2 billion people [1]. With the prospect of people living longer there has been concerted efforts to provide healthcare at appropriate times to allow the elderly to live as independently as possible, for as long as possible [2], [3]. Remote monitoring of older people in their homes may enable early detection of some of the problems associated with old age, such as propensity to falls and social isolation along with decline in cognitive function. Once Mild Cognitive Impairment (MCI) has set in it is usually too late to implement any cognitive training schema that could help. The biggest limiting factor to independence in the elderly is impaired cognitive function and its consequences. Such consequences include: accident proneness (falls, burns, bruising and cuts), self-neglect (missed medication, poor nutrition, poor hygiene), loss of initiative, diminished repertoire of activities and low mood. The assessment of cognitive function is expensive and labour intensive and hence non-viable for all but a tiny fraction of the elderly. In addition the elderly is a population traditionally highly averse to using new technology, which makes technical solutions to the assessment of cognitive function more difficult. Speech may provide a cost effective, easily implementable means of monitoring and assessing cognitive function. The disproportionate vulnerability of the brain’s frontal lobe to aging means that attention and executive function predominate in cognitive impairment having major detrimental effects on memory, attention, planning, initiative, mood vigilance as well as self-, safety-, and environmental awareness. One of the standard clinic tools used by psychologists for assessing cognitive function is the Mini Mental State Examination (MMSE) [5]. This consists of a questionnaire that rates cognitive function out of 30. MMSE scores of less than 27 are generally considered as possibly cognitively impaired by clinicians. Previous studies have found spoken language markers to be indicative of Mild Cognitive Impairment (MCI) [6]. Roark et al found standardized pause rate (the ratio of words uttered to the number of pauses uttered) to be statistically different for two sets of elderly speakers; using a Clinical Dementia Rating (CDR), the cohort was separated into those with MCI and those without. While other speech duration parameters were investigated in the study including verbal rate (number of words per second), phonation rate (proportion of total time spent speaking) and mean duration of pauses, none of these were found be significantly different for the two populations. Speech has also been shown to be an indicator of other neuro-degenerative diseases, such as schizophrenia [7]. Stassen et al looked at temporal parameters extracted from read speech from known sufferers of the disease and matched controls. In this case the correlation with this type of cognitive impairment and temporal features such as mean pause duration is attributed to the negative symptoms of the disease. These include affective flattening, blunted affect emotional dullness, poverty of speech and psycho-motor retardation. A recent study carried out by Lieberman et al [8], investigated audio recorded from climbers ascending Mount Everest. The effect of altitude on the cognitive function of climbers was used to simulate that experienced by astronauts in space. Astronauts' cognition is impacted by hypoxic and cosmic ray-induced insult to the brain, as well as degraded cognitive performance resulting from task difficulty. The authors found acoustic measures of temporal characteristics of speech can be used to monitor cognitive impairment. This study focuses on identifying the onset of cognitive decline from features that can readily be extracted from speech. Experimentation includes investigating speech from a temporal feature perspective and from an acoustic perspective. Speech is a non-intrusive means of data collection and speech centred applications, particularly telephony based, can be implemented using technology that is familiar to the potential users.

In phoneme recognition experiments, it was found that approximately 75% of misclassified frames were assigned labels within the same broad phonetic group (BPG). While the phoneme can be described as the smallest distinguishable unit of speech, phonemes within BPGs contain very similar characteristics and can be easily confused. However, different BPGs, such as vowels and stops, possess very different spectral and temporal characteristics. In order to accommodate the full range of phonemes, acoustic models of speech recognition systems calculate input features from all frequencies over a large temporal context window. A new phoneme classifier is proposed consisting of a modular arrangement of experts, with one expert assigned to each BPG and focused on discriminating between phonemes within that BPG. Due to the different temporal and spectral structure of each BPG, novel feature sets are extracted using mutual information, to select a relevant time-frequency (TF) feature set for each expert. To construct a phone recognition system, the output of each expert is combined with a baseline classifier under the guidance of a separate BPG detector. Considering phoneme recognition experiments using the TIMIT continuous speech corpus, the proposed architecture afforded significant error rate reductions up to 5% relative

Information about person identity is multimodal. Yet, most person-recognition systems limit themselves to only a single modality, such as facial appearance. With a view to exploiting the complementary nature of different modes of information and increasing pattern recognition robustness to test signal degradation, we developed a multiple expert biometric person identification system that combines information from three experts: audio, visual speech, and face. The system uses multimodal fusion in an automatic unsupervised manner, adapting to the local performance (at the transaction level) and output reliability of each of the three experts. The expert weightings are chosen automatically such that the reliability measure of the combined scores is maximized. To test system robustness to train/test mismatch, we used a broad range of acoustic babble noise and JPEG compression to degrade the audio and visual signals, respectively. Identification experiments were carried out on a 248-subject subset of the XM2VTS database. The multimodal expert system outperformed each of the single experts in all comparisons. At severe audio and visual mismatch levels tested, the audio, mouth, face, and tri-expert fusion accuracies were 16.1%, 48%, 75%, and 89.9%, respectively, representing a relative improvement of 19.9% over the best performing expert

The development of a system for automatically sorting a database of shoeprint images based on the outsole pattern in response to a reference shoeprint image is presented. The database images are sorted so that those from the same pattern group as the reference shoeprint are likely to be at the start of the list. A database of 476 complete shoeprint images belonging to 140 pattern groups was established with each group containing two or more examples. A panel of human observers performed the grouping of the images into pattern categories. Tests of the system using the database showed that the first-ranked database image belongs to the same pattern category as the reference image 65 percent of the time and that a correct match appears within the first 5 percent of the sorted images 87 percent of the time. The system has translational and rotational invariance so that the spatial positioning of the reference shoeprint images does not have to correspond with the spatial positioning of the shoeprint images of the database. The performance of the system for matching partial-prints was also determined.

The paper presents the classification performance of an automatic classifier of the electrocardiogram (ECG) for the detection of normal, premature ventricular contraction and fusion beat types. Both linear discriminants and feedforward neural networks were considered for the classifier model. Features based on the ECG waveform shape and heart beat intervals were used as inputs to the classifiers. Data was obtained from the MIT-BIH arrhythmia database. Cross-validation was used to measure the classifier performance. A classification accuracy of 89% was achieved which is a significant improvement on previously published results.

This study developed a system of automatic classification of shoeprint images into groups belonging to the same sole pattern. When presented with an image of a new shoeprint the system displays a ranked sequence of shoeprint images from the database. The shoeprint images are ranked from best match to worst match in terms of the pattern of the shoeprint. For this study a database of 503 shoeprint images belonging to 139 pattern groups was established with each group containing 2 or more examples. The pattern grouping was performed by a panel of human experts. This designed system is a fully automatic method and functions with minimum user intervention. Tests of the system have shown that the first shoeprint image displayed is a correct match 54% of the time and that a correct match appears within the first 5% of displayed shoeprints 75% of the time. The system has translational and rotational invariance so that the spatial positioning of the new shoeprint images does not have to correspond with the spatial positioning of the shoeprint images of the database.

Information concerning the identity of subword units such as phones cannot easily be pinpointed because it is broadly distributed in time and frequency.Continuing earlier work, we use Mutual Information as measure of the usefulness of individual time-frequency cells for various speech classification tasks, using the hand-annotations of the TIMIT database as our ground truth.Since different broad phonetic classes such as vowels and stops have such different temporal characteristics, we examine mutual information separately for each class, revealing structure that was not uncovered in earlier work; further structure is revealed by aligning the time-frequency displays of each phone at the center of their hand-marked segments, rather than averaging across all possible alignments within each segment.Based on these results, we evaluate a range of vowel classifiers over the TIMIT test set and show that selecting input features according to the mutual information criteria can provides a significant increase in classification accuracy.

A new method is proposed to automatically segment out a person's face from a given sequence of images that consists of a head-and-shoulder view, using the fast marching level set approach. The method proposed involves a fast, reliable and computationally efficient algorithm, which exploits the colour information in an image to segment the face region. The colour information is derived and used to classify pixels in the image into different levels depending on the value of the colour information for that pixel. Once this is achieved, a priority-labelled fast marching level set algorithm is utilised to locate the face region. The performance of the fast marching face-segmentation algorithm is illustrated by some simulation results carried out on head-and-shoulder test images.

Speechreading is the ability to understand a speaker’s thoughts by watching the movements of the face and body and by using the information provided by the situation and the language. People with normal hearing and the hearing impaired use speechreading to augment communication especially in noisy environments. Just as people learn this skill, machines can be trained to understand a speakers meaning. Audio-Visual Automatic Speech Recognition (AV ASR) systems use audio and visual information to recognize what has been ‘said’. The speech sounds and movements provided need not be standard speech sounds or movements. The system will provide recognition given audio information only, visual information only or both.

Audio-visual automatic speech recognition systems use visual information to enhance ASR systems in clean and noisy environments. This paper investigates a number of different visual feature extraction methods. It was observed that when performing visual speech recognition the visual feature vector requires a base level of detail for improved recognition. Geometric feature extraction provides lower recognition than pixel based methods due to the loss of characteristic speech information such as protrusion etc. Downsampling of images reduces visual recognition scores due to the loss of detail in the images. Also, the role of dynamic features was investigated for improved recognition. It was observed that static features alone outperform a combination of both static and dynamic features when restricting the dimension of the feature vector e.g. 50. This illustrates that the need for a certain level of detail in visual speech recognition is a higher priority than dynamic information. Once this base level of detail is attained the dynamic features should then be able to improve the recognition rate.

Twenty-live wavelet coefficients were selected as inputs and cross-validation used to estimate the classifier performance. An overall accuracy of 72.3% was achieved using a database of 500 ECG records independently classified into seven classes. This compared well with published cardiologist classification rates. By introducing a no-classification state, the accuracy increased to 7.9% with 80% of ECG records classified. The method presented here is not specific to the ECG domain and may easily be applied to other classification problems.

A novel segmentation algorithm for color video sequences using the level set technique is proposed. This algorithm is applied to the problem of automatic face region segmentation in video sequences. Given a target color range training set, a model of that color distribution is formed. An objective-function is then defined that seeks the boundaries of any regions in an image that match the target colors. This function is minimized using an implementation of the level set algorithm. Motion estimation greatly reduces the computation required for segmenting video sequences. Results are presented demonstrating the high level of accuracy and flexibility of the proposed segmentation algorithm.

This study compares the classification performance of feature sets formed from the discrete-wavelet-transform coefficients of different mother wavelets. Fifteen feature sets are calculated from three Daubechies wavelets, with the decomposition level varied between 3 and 7. Classification performance is optimised by using automatic feature selection and by combining classifications of multi-beat ECG information. Results show that the over all classification performance of the different feature sets was 71.6-74.2% and that the wavelet order and level had little influence on the over all performance. All quoted results are obtained from 10 runs of 10-fold cross-validation. An analysis of the automatically chosen features reveal that time-frequency bands in the vicinity of the QRS onset and the T-wave are consistently selected.

Traditionally online handwriting recognition (OHR) implementations use general-purpose processor architectures. The pre-processing step of OHR comprises regular array-based tasks such as normalisation, feature extraction and segmentation. Standard processor architectures cannot however efficiently support the varied arithmetic operations required by pre-processing. These tasks would seem ideally suited for custom hardware acceleration. CORDIC offers all the required elementary functions for pre-processing but is inefficient for linear mode operations (multiplication/division) due to its serial nature. A hybrid multiplier/CORDIC architecture is proposed in which a fast iterative multiplier/MAC shares hardware with a serial CORDIC unit. This multiplier retires 6b/cycle with minor additional hardware requirements. This hybrid offers improved general performance for signal-processing applications and is targeted at the pre-processing task of OHR. Performance results are included.