Automatic speech recognition systems transform speech audio data into text data, i.e., word sequences, as the recognition results. These word sequences are generally defined by the language model of the speech recognition system. Therefore, the capability of the speech recognition system to translate audio data obtained by typically pronouncing word sequences that are accepted by the language model into word sequences that are equivalent to the original ones can be regarded as a basic capability of the speech recognition systems. This work describes a testing method that checks whether speech recognition systems have this basic recognition capability. The method can verify the basic capability by performing the testing separately from recognition robustness testing. It can also be fully automated. We constructed a test automation system and evaluated though several experiments whether it could detect defects in speech recognition systems. The results demonstrate that the test automation system can effectively detect basic defects at an early phase of speech recognition development or refinement.

This paper describes a novel framework for creating a parser to process and analyze texts written in a “partially structured” natural language. In many projects, the contents of document artifacts tend to be described as a mixture of formal parts (i.e. the text constructs follow specific conventions) and parts written in arbitrary free text. Formal parsers, typically defined and used to process a description with rigidly defined syntax such as program source code are very precise and efficient in processing the formal part, while parsers developed for natural language processing (NLP) are good at robustly interpreting the free-text part. Therefore, combining these parsers with different characteristics can allow for more flexible and practical processing of various project documents. Unfortunately, conventional approaches to constructing a parser from multiple small parsers were studied extensively only for formal language parsers and are not directly applicable to NLP parsers due to the differences in the way the input text is extracted and evaluated. We propose a method to configure and generate a combined parser by extending an approach based on parser combinator, the operators for composing multiple formal parsers, to support both NLP and formal parsers. The resulting text parser is based on Parsing Expression Grammars, and it benefits from the strength of both parser types. We demonstrate an application of such combined parser in practical situations and show that the proposed approach can efficiently construct a parser for analyzing project-specific industrial specification documents.

We describe a new approach to modeling and analyzing software development documents that are typically written using conventional office applications. Our approach brings automation to content extraction, quality checking and measurement of massive document artifacts that tend to be handled by labor-intensive manual work in industry today. Rather than seeking an approach based on creation or rewriting of contents using more rigid, machine-friendly representations such as standardized formal models and restricted languages, we provide a method to deal with the diversity of document artifacts by making use of project-specific formalism that exists in target documents. We demonstrate that such project-specific formalism often tends to "naturally" exist at syntactic levels, and it is possible to define a "document model", a logical data representation gained by transformation rule from the physical, syntactic structure to the logical, semantic structure. With this transformation, various quality checking rules for completeness, consistency, traceability, etc., are realized by evaluating constraints for data items in the logical structure, and measurement of these quality aspects is automated. We developed a tool to allow a user to easily define document models and checking rules, and provide the insights on transformations when defining document models for various industry specification documents written in word processor files, spreadsheets and presentations. We also demonstrate the use of natural language processing can improve document modeling and quality checking by compensating for a weakness of formalism and applying analysis to specific parts of the target documents.

Igarashi and Kobayashi have proposed a general type system for checking whether resources such as files and memory are accessed in a valid manner. Their type system is, however, for call-by-value λ-calculus with resource primitives, and does not deal with non-functional primitives such as exceptions and pointers. We extend their type system to deal with exception primitives and prove soundness of the type system. Dealing with exception primitives is especially important in practice, since many resource access primitives may raise exceptions. The extension is non-trivial: While Igarashi and Kobayashi's type system is based on linear types, our new type system is a combination of linear types and effect systems. We also report on a prototype analyzer based on the new type system.