We believe teaching elementary school students to reason about programs is as important as teaching them to write programs. To facilitate development of this skill in young children one must choose a developmentally appropriate domain. Microsoft's Kodu Game Lab is a pattern-matching rule-based language whose semantics is significantly different than Scratch or Python. We chose Kodu because one can write non-trivial programs in two to four lines, and analyzing these programs is within the abilities of a typical 8 year old. Reasoning about programs requires students to understand the structure of code. The approach we're advocating is analogous to sentence diagramming, where one starts with a sequence of words and develops a representation of their syntactic and semantic relationships. One can similarly analyze Kodu programs by characterizing rules and recognizing relationships between rules. In this poster we describe "couplets", an analysis technique that reveals the presence within a program of an important Kodu design pattern called Pursue and Consume. Using this technique leads to accurate predictions about program behavior, and uncovers bugs if the pattern is not fully realized. As part of a study of 40 third graders who were learning Kodu, we provided brief instruction in the couplets technique. We found that they were able to apply couplets to 3-4 line programs and answer prediction questions with a roughly 85% success rate. Our results demonstrate that elementary school children can learn to reason abstractly about programs if given the right mental tools.

One use of text-to-speech synthesis (TTS) is as a component of speech-to-speech translation systems. The output of automatic machine translation (MT) can vary widely in quality, however. A synthetic voice that is extremely intelligible on naturally-occurring text may be far less intelligible when asked to render text that is automatically generated. In this paper, we compare the quality of synthesis of naturally-occurring text and its MT counterpart. We find that intelligibility of TTS on MT output is significantly lower than on either naturally-occurring text or semantically unpredictable sentences, and explore the reasons why.

This paper addresses the generation and evaluation of foreign-accented speech in concatenative text-to-speech (TTS) synthesis. We describe three possible methods of building a Spanish-accented English voice, and evaluate and compare them with respect to preference, intelligibility, and smoothness. Effects of speaking rate and content are also examined. It is found that although using an unmodified Spanish voice to read English text is possible, the result is not highly intelligible. With some modifications to the linguistic model, a relatively high level of comprehensibility and smoothness can be achieved, not differing widely from ratings given to a native voice at a comparable stage of development. Listeners in perceptual experiments were very consistent in their preference rankings of the three voices, showing that differences in voicebuilding method are both detectable and contribute to synthesis quality. 1.

... of synthesis of Arabic, a language that has shot to prominence in the past few years, and synthesis on a handheld device, realization of which presents difficult software engineering problems. Our system was developed in conjunction with the DARPA BABYLON project, and has been integrated with English synthesis, English and Arabic ASR, and machine translation on a single off-the-shelf PDA. We present

This paper describes a working two-way speech-to-speech translation system that runs in near real-time on a consumer handheld computer. It can translate from English to Arabic and Arabic to English in the domain of medical interviews. We describe the general architecture and frameworks within which we developed each of the components: HMM-based recognition, interlingua translation (both rule and statistically based), and unit selection synthesis.

Native and non-native use of language differs, depending on the proficiency of the speaker, in clear and quantifiable ways. It has been shown that customizing the acoustic and language models of a natural language understanding system can significantly improve handling of non-native input; in order to make such a switch, however, the nativeness status of the user must be known. In this paper, we show how the recognition hypothesis can be used to predict with very high accuracy whether the speaker is native. Effectiveness of both word-based and phone-based classification are evaluated, and a discussion of the primary discriminative features is presented. In an LVCSR system in which users are both native and non-native, we have achieved a 15.6% relative decrease in word error rate by integrating this classification method with speech recognition.

As speech recognition systems improve in quality, they become attractive as components in applications which will be used by non-native speakers, both applications designed specifically for language learners and those intended for general use. Recognizer performance on speakers who are not fluent in the language they are speaking, however, is often poor. A number of factors contribute to recognition failure for non-native speakers; pronunciation, lexical choice, and syntactic structure are a few of the elements of speech that set native and non-native speakers apart. In this paper we examine the character of non-native speech, both spontaneous and read, describing how features that are known to be important in recognition system development compare with those of native speakers.

We have tested a fixed version of the Fluency foreign language pronunciation trainer to determine whether users actually improve their pronunciation through its use. The Fluency system uses the CMU SPHINX II speech recognizer to pinpoint pronunciation errors and then gives suggestions as to how to correct them. Users in the experiment we will describe in this paper were to learn to pronounce the voiced and voiceless interdental fricatives /" / and /h/, as in "thin" and "that." The choice of these two phones allowed us to attract users from several native languages. In this paper, we present the results of this study, fully describing the test situation and drawing conclusions about Fluency as a testbed.

Building modern speech and language systems currently requires large data resources such as texts, voice recordings, pronunciation lexicons, morphological decomposition information and parsing grammars. Based on a study of the most important differences between language groups, we introduce approaches to efficiently deal with the enormous task of covering even a small percentage of the world's languages. For speech recognition, we have reduced the resource requirements by applying acoustic model combination, bootstrapping and adaption techniques. Similar algorithms have been applied to improve the recognition of foreign accents. Segmenting language into appropriate units reduces the amount of data required to robustly estimate statistical models. The underlying morphological principles are also used to automatically adapt the coverage of our speech recognition dictionaries with the Hypothesis-Driven Lexical Adaptation (HDLA) algorithm. This reduces the out-of-vocabulary problems encountered in agglutinative languages. Speech recognition results are reported for the read GlobalPhone database and some broadcast news data. For speech translation, using a task-oriented Interlingua allows to build a system with N languages with linear, rather than quadratic effort. We have introduced a modular grammar design to maximize reusability and portability. End-to-end translation results are reported on a travel-domain task in the framework of C-STAR.

When developing a speech recognition system, one must start by deciding what the units to be recognized should be. This is for the most part a straightforward choice in the case of word-based languages such as English, but becomes an issue even in handling languages with a complex compounding system like German; with an agglutinative language like Japanese, which provides no spaces in written text, the choice is not at all obvious. Once an appropriate unit has been determined, the problem of consistently segmenting transcriptions of training data must be addressed. This paper describes a method for learning a lexicon from a training corpus which contains no word-level segmentation, applied to the problem of building a Japanese speech recognition system. We show not only that one can satisfactorily segment transcribed training data automatically, avoiding human error, but also that our system, when trained with the automatically segmented corpus, showed a significant improvement in recognition performance.

JANUS-II is a research system to design and test components of speech-to-speech translation systems as well as a research prototype for such a system. We focus on two aspects of the system: (1) the new features of the speech recognition component JANUS-SR, and (2) the end-to-end performance of JANUS-II, including a comparison of two machine translation strategies used for JANUS-MT (PHOENIX and GLR*).

Attempts at discourse processing of spontaneously spoken dialogue face several difficulties: multiple hypotheses that result from the parser's attempts to make sense of the output from the speech recognizer, ambiguity that results from segmentation of multi-sentence utterances, and cumulative error -errors in the discourse context which cause further errors when subsequent sentences are processed.In this paper we will describe our robust parsers, our procedures for segmenting long utterances, and two approaches to discourse processing that attempt to deal with ambiguity and cumulative error.

As part of the JANUS speech-to-speech translation project, the authors have developed a robust translation system based on the information structures inherent to the task being performed. The basic premise is that the structure of the information to be transmitted is largely independent of the language used to encode it. The system performs no syntactic analysis; speaker utterances are parsed into semantic chunks, which can be strung together without grammatical rules, and passed through a simple template-based translation module. The authors have achieved encouraging coverage rates on English, German and Spanish input with English, German and Spanish output.