The wide availability of signal processing and language tools to extract structured data from raw content has created a new opportunity for the processing of structured signals. In this work, we explore models for the simulation and recognition of scenarios-i.e., time sequences of structured data. For simulation, we construct two models-hidden Markov models (HMMs) and event dependency graphs. Combined, these two simulation methods allow the specification of dependencies in event ordering, simultaneous execution of multiple scenarios, and evolving networks of data. For scenario recognition, we consider the application of multi-grained HMMs. We explore, in detail, mismatch between training scenarios and simulated test scenarios. The methods are applied to terrorist scenario detection with a simulation coded by a subject matter expert.

We apply statistical relational learning to a database of criminal and terrorist activity to predict attributes and event outcomes. The database stems from a collection of news articles and court records which are carefully annotated with a variety of variables, including categorical and continuous fields. Manual analysis of this data can help inform decision makers seeking to curb violent activity within a region. We use this data to build relational models from historical data to predict attributes of groups, individuals, or events. Our first example involves predicting social network roles within a group under a variety of different data conditions. Collective classification can be used to boost the accuracy under data poor conditions. Additionally, we were able to predict the outcome of hostage negotiations using models trained on previous kidnapping events. The overall framework and techniques described here are flexible enough to be used to predict a variety of variables. Such predictions could be used as input to a more complex system to recognize intent of terrorist groups or as input to inform human decision makers.

The intelligibility of speech transmitted through low-rate coders is severely degraded when high levels of acoustic noise are present in the acoustic environment. Recent advances in nonacoustic sensors, including microwave radar, skin vibration, and bone conduction sensors, provide the exciting possibility of both glottal excitation and, more generally, vocal tract measurements that are relatively immune to acoustic disturbances and can supplement the acoustic speech waveform. We are currently investigating methods of combining the output of these sensors for use in low-rate encoding according to their capability in representing specific speech characteristics in different frequency bands. Nonacoustic sensors have the ability to reveal certain speech attributes lost in the noisy acoustic signal; for example, low-energy consonant voice bars, nasality, and glottalized excitation. By fusing nonacoustic low-frequency and pitch content with acoustic-microphone content, we have achieved significant intelligibility performance gains using the DRT across a variety of environments over the government standard 2400-bps MELPe coder. By fusing quantized high-band 4-to-8-kHz speech, requiring only an additional 116 bps, we obtain further DRT performance gains by exploiting the ear's insensitivity to fine spectral detail in this frequency region.

We present highlights from three experiments that test the readability of current state-of-the art system output from: (1) an automated English speech-to-text (SST) system; (2) a text-based Arabic-to-English machine translation (MT) system; and (3) an audio-based Arabic-to-English MT process. We measure readability in terms of reaction time and passage comprehension in each case, applying standard psycholinguistic testing procedures and a modified version of the standard defense language proficiency test for Arabic called the DLPT*. We learned that: (1) subjects are slowed down by about 25% when reading system STT output; (2) text-based MT systems enable an English speaker to pass Arabic Level 2 on the DLPT*; and (3) audio-based MT systems do not enable English speakers to pass Arabic Level 2. We intend for these generic measures of readability to predict performance of more application-specific tasks.

The estimation of speech parameters and the intelligibility of speech transmitted through low-rate coders, such as MELP (mixed excitation linear prediction), are severely degraded when there are high levels of acoustic noise in the speaking environment. The application of nonacoustic and nontraditional sensors, which are less sensitive to acoustic noise than the standard microphone, is being investigated as a means to address this problem. Sensors being investigated include the general electromagnetic motion sensor (GEMS) and the physiological microphone (P-mic). As an initial effort in this direction, a multisensor MELPe coder (MELP coder with the addition of a noise preprocessor) using parameter substitution has been developed, where pitch and voicing parameters are obtained from GEMS and P-Mic sensors, respectively, and the remaining parameters are obtained as usual from a standard acoustic microphone. This parameter substitution technique is shown to produce significant and promising DRT (diagnostic rhyme test) intelligibility improvements over the standard 2400 bps MELPe coder in several high-noise military environments. Further work is in progress aimed at utilizing the nontraditional sensors for additional intelligibility improvements and for more effective lower-rate coding in noise.

This paper presents a novel information retrieval system that includes 1) the addition of concepts to facilitate the identification of the correct word sense, 2) a natural language query interface, 3) the inclusion of weights and penalties for proper nouns that build upon the Okapi weighting scheme, and 4) a term clustering technique that exploits the spatial proximity of search terms in a document to further improve the performance. The effectiveness of the system is validated by experimental results.

We discuss the security implications of different techniques used in adaptive audio and video distribution. Several sources of variability in the network make it necessary for applications to adapt. Ideally, each receiver should receive media quality commensurate with the capacity of the path leading to it from each sender. Several different techniques have been proposed to provide such adaptation. We discuss the implications of each technique for confidentiality, authentication, integrity, and anonymity. By coincidence, the techniques with better performance also have better security properties.

Telegraphic messages with numerous instances of omission pose a new challenge to parsing in that a sentence with omission causes a higher degree of ambiguity than a sentence without omission. Misparsing induced by omissions has a far-reaching consequence in machine translation. Namely, a misparse of the input often leads to a translation into the target language which has incoherent meaning in the given context. This is more frequently the case if the structures of the source and target languages are quite different, as in English and Korean. Thus, the question of how we parse telegraphic messages accurately and efficiently becomes a critical issue in machine translation. In this paper we describe a technical solution for the issue, and present the performance evaluation of a machine translation system on telegraphic messages before and after adopting the proposed solution. The solution lies in a grammar design in which lexicalized grammar rules defined in terms of semantic categories and syntactic rules defined in terms of part-of-speech are utilized together. The proposed grammar achieves a higher parsing coverage without increasing the amount of ambiguity/misparsing when compared with a purely lexicalized semantic grammar, and achieves a lower degree of ambiguity/misparses without decreasing the parsing coverage when compared with a purely syntactic grammar.

A spoken language system combines speech recognition, natural language processing and human interface technology. It functions by recognizing the person's words, interpreting the sequence of words to obtain a meaning in terms of the application, and providing an appropriate response back to the user. Potential applications of spoken language systems range from simple tasks, such as retrieving information from an existing database (traffic reports, airline schedules), to interactive problem solving tasks involving complex planning and reasoning (travel planning, traffic routing), to support for multilingual interactions. We examine eight key areas in which basic research is needed to produce spoken language systems: (1) robust speech recognition; (2) automatic training and adaptation; (3) spontaneous speech; (4) dialogue models; (5) natural language response generation; (6) speech synthesis and speech generation; (7) multilingual systems; and (8) interactive multimodal systems. In each area, we identify key research challenges, the infrastructure needed to support research, and the expected benefits. We conclude by reviewing the need for multidisciplinary research, for development of shared corpora and related resources, for computational support and far rapid communication among researchers. The successful development of this technology will increase accessibility of computers to a wide range of users, will facilitate multinational communication and trade, and will create new research specialties and jobs in this rapidly expanding area.>

The ARPA Spoken Language Technology and Applications Day (SLTA'93) was a special workshop which presented a set of live, state-of-the-art demonstrations of speech recognition and spoken language understanding systems. The purpose of this paper is to provide perspective on current opportunities for applications of spoken language technology by summarizing the demonstrations and the related applications which they can enable, and reviewing the applications opportunities and needs cited by panelists and other members of the user community.>

The author presents a study of military applications of advanced speech processing technology which includes three major elements: (1) review and assessment of current efforts in military applications of speech technology. (2) identification of opportunities for future military applications of advanced speech technology, and (3) identification of problem areas where research in speech processing is needed to meet application requirements, and of current research thrusts which appear promising. Opportunities for advanced applications are identified by means of descriptions of several generic systems which would be possible with advances in speech technology and in system integration. These generic systems include an integrated multirate voice/data communications terminal, an interactive speech enhancement system, a voice-controlled pilot's associate system, advanced air traffic control training systems, a battle management command and control support system with spoken natural language interface, and a spoken language translation system. In the applications discussed, various spectral analysis and digitization techniques are teamed up with artificial intelligence schemes to allow a machine to recognize continuous-speech sentences with vocabularies running to thousands of words.>

A speaker activity detector is described. It takes cochannel speech as input and labeling intervals of the input as target-only, jammer-only, or two-speaker (target and jammer). The algorithms applied are borrowed primarily from speaker recognition, thereby allowing one to use speaker-dependent test-utterance-independent information in a front-end for cochannel talker interference suppression. Parameters studied include classifier choice (vector quantization vs. Gaussian), training method (unsupervised vs. supervised), test utterance segmentation (uniform vs. adaptive), and training and testing target-to-jammer ratios. Using analysis interval lengths of 100 ms, performance reached 80% correct detection.>

A description is given of the evaluation of a simulated suppression system performing ideal suppression of a jammer signal, given the voicing states (voiced, unvoiced, silent) of the target and jammer speech as a function of time and the isolated target and jammer speech waveforms. By applying suppression to select regions of jammer speech as a function of the voicing states of the target and jammer, and by measuring the intelligibility of the resulting jammer-suppressed cochannel speech, it is possible to identify those regions of cochannel speech on which interference suppression most improves intelligibility. Such results can help focus algorithm development efforts.>

A series of experiments were performed to determine (1) the effects of using an energy-based endpoint detector and a conventional isolated word recognition system when the input speech is noisy and (2) the effects of placing a noise suppression prefilter in tandem with the word recognizer in an attempt to remove the noise prior to recognition. It was found that the system consisting of the prefilter working in tandem with the word recognizer increased word recognition accuracy.

The integration of digital voice with data in a common packet-switched network system offers a number of potential benefits, including reduced systems cost through sharing of switching and transmission resources, flexible internetworking among systems utilizing different transmission media, and enhanced services for users requiring access to both voice and data communications. Issues which it has been necessary to address in order to realize these benefits include reconstitution of speech from packets arriving at nonuniform intervals, maximization of packet speech multiplexing efficiency, and determination of the implementation requirements for terminals and switching in a large-scale packet voice/data system. A series of packet speech systems experiments to address these issues has been conducted under the sponsorship of the Defense Advanced Research Projects Agency (DARPA). In the initial experiments on the ARPANET, the basic feasibility of speech communication on a store-and-forward packet network was demonstrated. Techniques were developed for reconstitution of speech from packets, and protocols were developed for call setup and for speech transport. Later speech experiments utilizing the Atlantic packet satellite network (SATNET) led to the development of techniques for efficient voice conferencing in a broadcast environment, and for internetting speech between a store-and-forward net (ARPANEI) and a broadcast net (SATNET). Large-scale packet speech multiplexing experiments could not be carried out on ARPANET or SATNET where the network link capacities severely restrict the number of speech users that can be accommodated. However, experiments are currently being carried out using a wide-band satellite-based packet system designed to accommodate a sufficient number of simultaneous users to support realistic experiments in efficient statistical multiplexing. Key developments to date associated with the wide-band experiments have been 1) techniques for internetting via voice/data gateways from a variety of local access networks (packet cable, packet radio, and circuit-switched) to a long-haul broadcast satellite network and 2) compact implementations of packet voice terminals with full protocol and voice capabilities. Basic concepts and issues associated with packet speech systems are described. Requirements and techniques for speech processing, voice protocols, packetization and reconstitution, conferencing, and multiplexing are discussed in the context of a generic packet speech system configuration. Specific experimental configurations and key packet speech results on the ARPANET, SATNET, and wide-band system are reviewed.

Voice communication networks have traditionally been designed to provide either analog signal paths or fixed-rate synchronous digital connections between individual subscribers. These designs were aimed at accommodating the "streamlike" character of speech, which has traditionally been considered to flow from source to destination at a more or less constant rate. By way of contrast, interactive and computer-to-computer data transactions tend to be "bursty" in nature, and this has given rise to the development of packet-switching methods for data communications. The dichotomous nature of these two major traffic classes and the apparent conflict between the types of network services they require has resulted in the deployment of separate military communications facilities for voice and data. A challenge in the design of future systems is to achieve overall economy and flexibility in the allocation of resources via the efficient integration of both traffic types in common network facilities. This paper summarizes a number of advanced concepts for switching and flow control of combined voice and data traffic in integrated environments. Performance characteristics are described based on analysis results and computer simulation studies for both multilink terrestrial and broadcast satellite network topologies.

Results are developed for data traffic performance in an integrated multiplex structure which includes circuit-switching for voice and packet-switching for data. The results are obtained both through simulation and analysis, and show that excessive data queues and delays will build up under heavy loading conditions. These large data delays occur during periods of time when the voice traffic load through the multiplexer exceeds its statistical average. A variety of flow control mechanisms to reduce data packet delays are investigated. These mechanisms include control of voice bit rate, limitation of the data buffer, and combinations of voice rate and data buffer control. Simulations indicate that these flow control mechanisms provide substantial improvements in system performance.

A packetized speech multiplexer differs from a circuitswitched TASI system in that the presence of a packet buffer allows a tradeoff where the TASI advantage can be increased at a cost in packet delay. This tradeoff is investigated via a simulation. Results are presented to show the relations between TASI advantage and delay, for both an average delay criterion and a maximum delay criterion. It is shown that, particularly for the case where small numbers of talkers are multiplexed, the packetized system offers significant improvements in TASI advantage over the conventional circuit-switched multiplexer, at modest costs in packet delay.

The cutout fraction in a TASI system is shown to be\phi=\frac{1}{np}\sum_{k=c+1}^n (k-c) n\choose k p^{k}(1-p)^{n-k}wherenis the number of sources,cis the number of channels, andpis the probability that a source is issuing a talkspurt at a random time. This result is shown to hold independently of the probability density function of talkspurt duration. The same formula is shown to apply to the fraction of packets lost in a packet-switched link with a transmission capacity ofcpackets every Tpseconds, where Tpis the interval between packet generations for an individual source during talkspurt, and where no packet is queued for a time longer than Tp. In addition, a simple Markov birth-death model is presented for the random processa(t)representing the number of talkers issuing talkspurts at a given time.