We compare the F0 output and chunking size of speech units of L1 English, TW L2 English and L1 Mandarin to see what some of the major intrinsic prosodic difference could be, what prosodic features could account for TW L2 English and in what way prosodic transfer occurs. Results show that the fundamental prosodic difference of the two languages is how in the pitch domain English requires sharper high/low contrast by higher-level prosodic units but less such contrast in accentuating lower-level prosodic units whereas Mandarin patterns are the exact opposite. Explanations are provided regarding how TW L2 English differs from L1, why prosodic transfer merits detailed analysis, and why mastering English prosody is especially difficult.

The present study examines perceived prosodic highlights in three genres of fluent continuous Mandarin to test (1) whether prosodic highlights are genre related, (2) how they interact with discourse structure, (3) how they signal information status, (4) whether systematic acoustic patterns could be obtained from speech data analysis, and (5) whether prosodic highlights is layered over to discourse structure. Results demonstrate that prosodic highlighting is genre related; distribution of key information can be attributed to linguistic content and communicative needs. Prosodic highlighting is an extra layer over discourse structure, the former signals key information while the latter underlying linguistic association.

The prosody of a sentence (utterance) when it appears in a discourse context differs substantially from when it is uttered in isolation. This paper addresses why paragraph is a discourse unit and discourse prosody is an intrinsic part of naturally occurring speech. Higher level discourse information treats sentences, phrases and their lower level units as sub-units and layers over them; and realized in patterns of global prosody. A perception based multi-phrase discourse prosody hierarchy and a parallel multi-phrase associative template were proposed to test discourse prosodic modulations. Results from quantitative modeling of speech data show that output discourse prosody can be derived through multiple layers of higher level modulations. The seemingly random occurrence of lower level prosodic units such as intonation variations is, in fact, systematic. In summary, abundant traces of global prosody can be recovered from the speech signal and accounted for; their patterns could help facilitate better understanding of spoken language processing.

This position paper highlights the importance of suprasegmental training in secondary language (L2) acquisition. Suprasegmental features are manifested in terms of acoustic cues and convey important information about linguistic and information structures. Hence, L2 learners must harness appropriate suprasegmental productions for effective communication. However, this learning process is influenced by well-established perceptions of sounds and articulatory motions in the primary language (L1). We propose to design and collect a corpus to support systematic analysis of L2 suprasegmental features. We lay out a set of carefully selected textual environments that illustrate how suprasegmental features convey information including part-ofspeech, syntax, focus, speech acts and semantics. We intend to use these textual environments for collecting speech data in a variety of Asian Englishes from non-native English speakers. Analyses of such corpora should lead to research findings that have important implications for language education, as well as speech technology development for computer-aided language learning (CALL) applications. Index Terms: L2 suprasegmental features, language education, prosody, stress, intonation.

This paper discusses discourse planning of pre-organized spontaneous narratives (SpnNS) in comparison with read speech (RS). F0 and tempo modulations are compared by speech paragraph size and discourse boundaries. The speaking rate of SpnNS from university classroom lecture is 2 to 3 times to that of RS by professionals; paragraph phrasing of SpnNS is 6 times that of RS. Patterns of paragraph association are distinct for SpnNS and RS. Sub-paragraph and paragraph units in RS are marked by distinct relative F0 resets and boundary pause duration, but by patterns of intensity contrasts in SpnNS instead. Consistent to both data sets is the finding that combined relative supra-segmental cues reflecting global prosodic properties are more discriminative to distinguish discourse boundaries than any fragments of singular cue, supporting higher-level discourse planning in the acoustic signals. We believe these findings can be directly applied to speech technology development.

We hypothesize that various prosody output styles can be predicted and simulated from one default base form by accounting for contributions from higher level information to cross-phrase prosodic relationship. Speech materials of four prosody styles were selected: (1.) Han and Tang poetry, (2.) Tang Ballads and Song poetry, (3.) Qin, Tang and Song classic prose and (4.) contemporary TV weather forecast. F0 contours were analyzed using the Fujisaki model, while quantitative analyses of predictions from layered-andcumulative contribution specified by the HPG (Hierarchical Prosodic phrase Grouping) framework [Tseng et al, 2004; 2005; 2006] were performed across styles and speakers. Results confirmed that higher level contribution is significant across style; contribution distribution patterns and style specific; more regular prosodic formats require more contribution from higher level; stylistic dynamics are predictable; and the HPG base form is indeed default.

This paper reports 3 perception experiments on intonation groups and the role of phrasal intonation in the organization of speech prosody. The goal is to help unlimited TTS achieve better naturalness. Experiments were also designed to compliment previous extensive analyses of speech data. Using the PRAAT software and removing segmental information humming experiments of extracted intonation groups ending in interrogative and declarative intonations in both complete and edited forms were used. Results showed that (1.)phrasal or sentential intonation contour is less significant for Mandarin, (2.) yes-no questions with utterance question particles are characterized by a rising pitch on the final syllable only, (3) the general higher register exhibited in yes-no questions without utterance final question particles is not the most salient cue for intonation, (4.) utterance final lengthening appears to be a salient perceptual cue for intonation identification, (5.) speech units larger than single sentences deserve more attention.

From the perspective of speech technology development for unlimited Mandarin Chinese TTS, two issues appear most impedimental: (1.) how to predict prosody from text, and (2.) how to achieve better naturalness for speech output. These impediments somewhat brought out the major pitfalls in related research, i.e., characteristics of Chinese connected speech and the overall rhythmic structure of speech flow. This paper discusses where the problems stem from and how some solutions could be found. We propose that for Mandarin, prosody research needs to include the following: (1.) characteristics of Mandarin connected speech that constitute the prosodic properties in speech flow, i.e., units and boundaries, (2.) scope and type of speech data collected, i.e., text other than isolated sentences, (3.) prosody in relation to speech planning, i.e., information other than lexical, syntactic and semantic, and (4.) an overall organization of prosody for speech flow, i.e., a framework that accommodate the above mentioned features.

This paper presents a set of corpus-based text-to-speech synthesis technologies for Mandarin Chinese. A large speech corpus produced by a single speaker is used, and the speech output is. synthesized from waveform units of variable lengths, with desired linguistic properties, retrieved from this corpus. Detailed methodologies were developed for designing "phonetically rich" and "prosodically rich" corpora by automatically selecting sentences from a large text corpus to include as many desired phonetic combinations and prosodic features as possible. Automatic phonetic labeling with iterative correction rules and automatic prosodic labeling with a multi-pass top-down procedure were also developed such that the labeling process for the corpora can be completely automatic. A hierarchical prosodic structure for an arbitrary desired text sentence is then generated based on the identification of different levels of break indices, and the prosodic feature sets and appropriate waveform units are finally selected and retrieved from the corpus, modified if necessary, and concatenated to produce the output speech. The special structure of Mandarin Chinese has been carefully considered in all these technologies, and preliminary assessments indicated very encouraging synthesized speech quality.

This paper aims to (1.) quantify possible correlation between syntactic structure and prosodic manifestation in Mandarin Chinese, (2.) explore to what extent such correlation could be predicted by syntactic structures and what may go beyond these correlations, and (3.) increase more operational characteristics to Mandarin prosodic structures.

The removal of noise from speech signals has applications ranging from speech enhancement for cellular communications to front ends for speech recognition systems. In this paper, we present a new nonlinear time-domain method called Noise-Regularized Adaptive Filtering. The approach is based on minimum mean-squared estimation using a modified cost function and allows designing both linear and nonlinear filters using only the observed noisy speech. 1. A GENERAL FRAMEWORK FOR MMSE ESTIMATION Given a noisy speech signal,

This paper describes an improved concatenative synthesis module for a Chinese text-to-speech system. The concatenated segments are on-line selected from a designed speech corpus that is precisely segmented with an improved version of HMMs. The selection criteria are the prosodic and contextual similarities between the units and the desired targets from the previous module of the TTS system. The TD-PSOLA modifies the prosodic parameters of the selected units, and three methods for unit concatenation are performed according to the types of the syllabic junctures. These types are classified with the knowledge from the phonetic observations of large amounts of speech data. The output speech is remarkably fluent and natural because the coarticulation effects cross syllabic boundaries are well modeled and less prosodic modification is needed for the TD-PSOLA.

This paper presents a multi-phase approach for fast spotting of large vocabulary Chinese keywords from a spontaneous Mandarin speech utterance using prosodic knowledge. Without searching through the whole utterance using large number of keyword models, the multi-phase framework proposed including some special scoring schemes provides very good efficiency by considering the monosyllable-based structure of Mandarin Chinese. This approach is therefore very fast due to very good boundary estimations and the deletion of most impossible syllable and keyword candidates using context independent models, and is also very accurate due to the carefully designed scoring processes. A task with 2611 keywords was tested. An inclusion rate of 85.79% for the top 10 candidates is attained, at a speed requiring only 1.2 times that of the utterance length on a Sparc 20 workstation.

This paper presents a new Chinese text-to-speech system that produces very natural and intelligible synthetic Mandarin speech based on part-of-speech analysis, prosodic modeling and non-uniform units. The distinguishing features and key technology for the system can be summarized as follows. (1) A text analysis module for word identification and tagging was developed based on part-of-speech modeling and using heuristic rules to achieve very high accuracy. (2) The required prosodic parameters for the synthetic speech are derived from a two-stage procedure. The prosodic structures of the input texts are first derived from a statistical model trained by a large speech database, and the prosodic parameters are then determined according to the structures. (3) A specially designed speech segments inventory constructed with non-uniform and pitch dependent units is used to improve the fluency and intelligibility of the system.

A set of new approaches is proposed to model the Mandarin syllables for accurate recognition with limited training data while specially considering the effect of tones, including improved initial values and state transition topologies, and making use of the durational cue. The results show that these approaches are very useful practically.

A key problem for today's speech synthesis technology is to automatically generate an appropriate hierarchical prosodic structure for text input and incorporate it into synthesized sptech[l][2].This paper presents a method for such a problem in Mandarin Chinese.This method uses a speech database for the training of a statistical model to generate the prosodic structure and determine prosodic parameten such as syllable duration, pause, energy and intonation.The experimental results show that an accuraq of 83.1% in the prediction of prosodic structure can be achieved.Furthermore, a Chinese text-to-pech system can be developed based on the proposed prosodic st"Te.

This paper presents a prototype prosodic-segment-based Mandarin dictation machine for the Chinese language with very large vocabulary. It accepts utterances continuous within a prosodic segment which is composed of one or a few word(s). It also possesses various on-line learning capabilities for fast adaptation to a new user in acoustic, lexical and linguistic levels. The overall system is implemented on an IBM/PC with an additional DSP card including a Motorola DSP 96002 chip. The word accuracy can achieve nearly 90% for a new user after he produces about 10 minutes of speech to train the system, and the accuracy can be further improved with the on-line learning functions.

This paper presents the first known results for complete recognition of continuous Mandarin speech for Chinese language with very large vocabulary but very limited training data. Although some isolated-syllable-based or isolated-word-based large-vocabulary Mandarin speech recognition systems have been successfully developed, a continuous-speech-based system of this kind has never been reported before. For successful development of this system, several important techniques have been used, including acoustic modeling of a set of sub-syllabic models for base syllable recognition and another set of context-dependent models for tone recognition, a multiple candidate searching technique based on a concatenated syllable matching algorithm to synchronize base syllable and tone recognition, and a word-class-based Chinese language model for linguistic decoding. The best recognition accuracy achieved is 88.69% for finally decoded Chinese characters, with 88.69%, 91.57%, and 81.37% accuracy for base syllables, tones, and tonal syllables respectively.

The first successfully implemented real-time Mandarin dictation machine, which recognizes Mandarin speech with very large vocabulary and almost unlimited texts for the input of Chinese characters into computers, is described. The machine is speaker-dependent, and the input speech is in the form of sequences of isolated syllables. The machine can be decomposed into two subsystems. The first subsystem recognizes the syllables using hidden Markov models. Because every syllable can represent many different homonym characters and form different multisyllabic words with syllables on its right or left, the second subsystem is needed to identify the exact characters from the syllables and correct the errors in syllable recognition. The real-time implementation is on an IBM PC/AT, connected to three sets of specially designed hardware boards on which seven TMS 320C25 chips operate in parallel. The preliminary test results indicate that it takes only about 0.45 s to dictate a syllable (or character) with an accuracy on the order of 90%.>

A set of improved tone concatenation rules to be used in a formant-based Mandarin Chinese text-to-speech system is presented. This system concatenates prestored syllables superimposed by additional tone patterns to obtain speech sentences for unlimited text, with the acoustic properties of each syllable modified by a set of synthesis rules. The tone concatenation rules are the most important among these synthesis rules, because they tell how the tone patterns for the syllables should be modified in an arbitrary sentence under various conditions of concatenating syllables of different tones on both sides. The improved tone concatenation rules are obtained empirically by carefully analyzing the tone pattern behavior under various tone concatenation conditions for many sentences in a database. A total of 14 representative tone patterns are defined for the five tones, and different rules about which pattern should be used under what kind of tone concatenation conditions are organized in detail. Preliminary subjective tests indicate that these rules actually give better synthesized speech for a formant-based Chinese text-to-speech system.>

A successfully implemented real-time Mandarin dictation machine which recognizes Mandarin speech with unlimited texts and very large vocabulary for the input of Chinese characters to computers is described. Isolated syllables including the tones are first recognized using specially trained hidden Markov models with special feature parameters. The exact characters are then identified from the syllables using a Markov Chinese language model. The real-time implementation is on an IBM PC/AT, connected to a set of special hardware boards on which ten TMS 320C25 chips operate in parallel. It takes only 0.45 s to dictate a character.>

In a long-term research project, the recognition of Mandarin speech for very large vocabulary and unlimited text is considered. Its first stage goal is to recognize the Mandarin syllables. In a previous paper, an initial/final two-phase recognition approach to recognize these very confusing syllables was proposed, in which each syllable is divided into initial and final parts and recognized separately, and efficient recognition techniques for the finals were proposed and discussed. This paper serves as a continuation and proposes an efficient system to recognize the Mandarin initials. In this system, a classification procedure is first used to categorize the unknown initials into two groups C1 and C2; different approaches are then separately applied and independently optimized to recognize C1 and C2. It is found that Finite State Vector Quantization (FSVQ) is very useful, whose two modified versions, Modified FSVQ (MFSVQ) and the Second Order FSVQ (SOFSVQ), can provide the best recognition performance for C1 and C2 by carefully adjusting a design parameter called characteristic interval. Experimental results show that a recognition rate of 94.1% to 94.7% can be achieved using this system. Such a design is accomplished by carefully considering the special characteristics of Mandarin syllables and initials.

An experimental Mandarin dictation machine for inputting Mandarin speech (spoken Chinese language) into computers is described. Because of the special characteristics of the Chinese language, syllables are chosen as the basic units for dictation. The machine is designed based on a hierarchical language recognition approach in which acoustic signals are first recognized as a sequence of syllables, possible word hypotheses are then formed from the syllables, and the complete sentences are finally obtained. This approach is implemented by two subsystems. The first recognizes the syllables using speech signal processing techniques, the second subsystem then identifies the exact characters from the syllable and corrects the errors in syllable recognition. The detailed syllable recognition algorithms, word formation rules, parser, grammar, and the syntactic checking algorithms are described. With newspaper text in the form of isolated syllables as input, the preliminary test results indicate that such a dictation machine is not only practically attractive, but technically feasible.>

The label-transition finite-state vector-quantization (FSVQ) algorithm is extensively explored to exhibit the power of finite-state machines for speech recognition. It is found that the FSVQ algorithm combined with special structural constraints can discriminate a finite set of candidates very successfully. All the consonant initials of isolated Mandarin monosyllables from designated speakers are used as the example vocabulary in the simulation. In addition to utilizing the first order memory provided by FSVQ on speech recognition, an experiment is conducted that expands the FSVQ to use the second-order memory and the dynamic relationship among the components of this three-vector group are used for recognition. The simulation results show that a slightly higher recognition rate (94.4%) is obtained with a consistent prediction interval.>

A long-term research project toward Mandarin speech recognition techniques for very large vocabulary and unlimited text is considered. By carefully examining the special structures of Chinese language, the first-stage goal is set to be the design of efficient techniques to recognize the finals of Mandarin syllables. In this paper, three special approaches to do this are proposed. The Segmental Model Approach defines the final models by dividing the finals into several segments according to the acoustic structures of the speech signals. The Three-pass Approach uses three consecutive passes to classify the finals into small sets and improve the recognition efficiency. The Multi-section Vector Quantization (MSVQ) Approach, on the other hand, significantly reduces the necessary computation time by incorporating the branch-and-bound algorithm and common codebook concept with the MSVQ techniques. Extensive computer simulations are performed first to optimize each approach by choosing the best set of parameters then to compare the performance of the three approaches. It was found that all the three approaches are very efficient in terms of relatively high recognition rate and short computation time, and the MSVQ Approach provides the highest recognition rate at the shortest computation time, thus it is most attractive.

An unrestricted Chinese text-to-speech system has been successfully constructed. The text input which is written in Mandarin Phonetic Symbols II (MPS II) instead of traditional Chinese orthography is analyzed by a simple parser. Then each syllable in the text is assigned proper duration and pitch, while its gain is modified, and its reflection coefficients interpolated or decimated. The stress, intonation and inter-syllable pause are also considered. In other words, a model of concatenating syllables of Mandarin Chinese based on simplified phonetic rules and some preliminary analysis of natural speech data is proposed. We believe that our model is sufficient compared with some more complicated approaches [1]. After editing the syllables according to our proposed synthesis rules, a continuous speech signal is synthesized, and then transmitted to speech interface card for speech output. Samples of synthesized speech by our text-to-speech system have been tested on subjects with satisfactory results. Evaluation of the system in intelligibility, comprehensibility and fluency has been conducted and an average of 96%, 95.9% and 75% respectively was reached.