With the increasing popularity of cloud computing, many notable quality of service (QoS)-aware service composition approaches have been incorporated in service-oriented cloud computing systems. However, these approaches are implemented without considering the energy and network resource consumption of the composite services. The increases in energy and network resource consumption resulting from these compositions can incur a high cost in data centers. In this paper, the trade-off among QoS performance, energy consumption, and network resource consumption in a service composition process is first analyzed. Then, a green service composition approach is proposed. It gives priority to those composite services that are hosted on the same virtual machine, physical server, or edge switch with end-to-end QoS guarantee. It fulfills the green service composition optimization by minimizing the energy and network resource consumption on physical servers and switches in cloud data centers. Experimental results indicate that, with comparisons to other approaches, our approach saves 20-50 percent of energy consumption and 10-50 percent of network resource consumption.

An Elastic Machine (EM) consists of networked fine-grained resources, such as CPU, memory, network and storage, dynamically composed from distributed resource pools offered by a RaaS (Resource-as-a-Service) cloud. EM allocation in RaaS clouds differs from VM placement in IaaS clouds in that it needs to consider the network conditions between the fine-grained resources. To address this problem, this paper presents a d-tree model to represent network conditions for both EM and resource pools and treat EM allocation as a tree packing problem. To solve this NP hard problem efficiently, this paper describes a tree packing framework that combines 12 approximate algorithms by using tree density to sort d-trees and virtual distance to filter d-trees. Using simulation tests and 6 quality measures, new algorithms that outperform the previous tree packing algorithm are discovered. Furthermore, the tests show that the algorithms that respect the network conditions outperform those that ignore them in most cases. Moreover, the top 2 algorithms are identified using a ranking function that combines the quality measures.

REST architectural style has become a prevalent choice for distributed resources, such as the northbound API of software-defined networking (SDN). As services often undergo frequent changes and updates, the corresponding REST APIs need to change and update accordingly. To allow REST APIs to change and evolve without breaking its clients, a REST API can be designed to facilitate hypertext-driven navigation and its related mechanisms to deal with structure changes in the API. This paper addresses the issues in hypertext-driven navigation in REST APIs from three aspects. First, we present REST Chart, a Petri-Net-based REST service description framework and language to design extensible REST APIs, and it is applied to cope with the rapid evolution of SDN northbound APIs. Second, we describe some important design patterns, such as backtracking and generator, within the REST Chart framework to navigate through large scale APIs in the RESTful architecture. Third, we present a client side differential cache mechanism to reduce the overhead of hypertext-driven navigation, addressing a major issue that affects the application of REST API. The proposed approach is applied to applications in SDN, which is integrated with a generalized SDN controller, SOX. The benefits of the proposed approach are verified in different conditions. Experimental results on SDN applications show that on average, the proposed cache mechanism reduces the overhead of using the hypertext-driven REST API by 66%, while fully maintaining the desired flexibility and extensibility of the REST API.

Although many current Cloud computing platforms are still based on virtual machines running on hyper visors, there are two trends towards a more flexible and efficient Cloud computing paradigm. One trend is the Resource-as-a-Service (RaaS) based Cloud, where the fine-grained resources can be rented at short time intervals. The other one is the container-based Cloud, where the lightweight containers replace the virtual machines. Both trends require a more flexible and efficient resource management framework to address the increased demands for scalability, heterogeneity, flexibility, deployment density, and efficiency. To provide such a resource management framework, this paper describes a REST service framework based on the concept of Resource-Oriented Network (RON) that decouples the resource representation plane, control plane, and data plane. Created by an automated top-down development process, RON provides uniform and cohesive REST APIs to manage the fine-grained resources for processes, tasks, jobs, containers, servers, and clusters, by encapsulating the underlying Linux resource control models. In addition, we present a Role-Based Access Control (RBAC) architecture to secure the access to RON in the Cloud. A prototype system has been implemented and our experimental results show that the approach is feasible and the performance is consistent with the client and server workloads.

To realize the potential benefits of SDN, tremendous amount of network information has to be collected, stored, processed and retrieved for the central routing and resource allocation optimization in near real-time. However, the current SDN-Open Flow controller and management applications did not address this issue properly and could not easily and efficiently retrieve and utilize the available analytics on-time. Their "network information bases" are rudimentary, and could not facilitate the required fast processing and accessing for required intelligent information for large networks. We present an innovative mechanism that architects such information into a multi-dimensional model, utilizing prevailing technologies in large scale real-time data processing, the advances in in-memory SQL and NoSQL and the Online Analytical Processing (OLAP). Data and their hidden characteristics and inter-relationships can then be stored in-memory or cached using key/values as in NoSQL, while more static information could be persisted into local file system or SQL database, and globally in a centralized SQL database (or a data warehouse) that can store large amounts of historical data that can be easily processed with OLAP or data mining for intelligence. Extensive experiments and comparison with traditional relational database or in-memory hashing table clearly demonstrated the feasibility, capability and performance advantage of the proposed approach.

REST Chart is a Petri-Net based XML modeling framework for REST API. This paper presents two important enhancements and extensions to REST Chart modeling - Hyperlink Decoration and Hierarchical REST Chart. In particular, the proposed Hyperlink Decoration decomposes resource connections from resource representation, such that hyperlinks can be defined independently of schemas. This allows a Navigation-First Design by which the important global connections of a REST API can be designed first and reused before the local resource representations are implemented and specified. Hierarchical REST Chart is a powerful mechanism to rapidly decompose and extend a REST API in several dimensions based on Hyperlink Decoration. These new mechanisms can be used to manage the complexities in large scale REST APIs that undergo frequent changes as in some large scale open source development projects. This paper shows that these new capabilities can fit nicely in the REST Chart XML with very minor syntax changes. These enhancements to REST Chart are applied successfully in designing and verifying REST APIs for software-defined-networking (SDN) and Cloud computing.

This paper presents a REST service composition framework based on functional programming with monads. It is motivated by the acute needs for dynamism and compos ability in concurrent control of large number of REST resources, especially in the areas of Cloud computing, software-defined networking (SDN), and Internet-of-Things/Web-of-Things. Dynamism gives us the ability to write a REST service composition program to invoke REST services whose identifications and operations are unknown at the design time, and compos ability gives us the ability to combine existing programs for different purposes. The key idea of our approach is to treat the primitives of a composition program, including resource identifications, operations, and control structures, as unknown functions and monads that can be dynamically composed from known functions and monads. We present the algebraic operators and rules that govern the compositions and its translation to XML, to combine the modularity and flexibility of functional programming with the portability and extensibility of XML. A prototype composition engine has been implemented and used to compose REST services in the Cloud computing domain. The initial experimental results indicate that the approach is feasible and promising.

Network virtualization is a promising technology for the next generation network, which is required to offer a more dynamic and flexible network infrastructure. Virtual network embedding plays a vital role in the resource allocation of network virtualization. Current virtual network embedding allocates resources in an exclusive and excessive manner. For example, the whole bandwidth amount of the virtual network's peak traffic demand is allocated to the virtual network with full availability. However, such excessive resource allocation may result in resource under-utilization, leading to high user cost and low carrier revenue. To address this problem, we propose a new dynamic resource pooling and trading mechanism. The proposed mechanism is formulated as a Stackelberg game, using bandwidth as an example of the resources. We compare the user cost and the carrier revenue under the proposed mechanism against those under the exclusive resource allocation schemes. Our results show that under certain conditions, the "win-win" situation, in which the user saves cost and the carrier increases its revenue, exists and the optimal Subgame Perfect Equilibrium (SPE) point can be found.

This paper describes a new and generic REST service composition framework that uses 6 algebraic rules to drive resource-resource compositions in different domains. The generalization is achieved by two techniques borrowed from Category Theory: 1) decomposition of a REST system into cohesive link and communication layers, and 2) uniform representation of resource interfaces and resource compositions as categorial links. Within this framework, we introduce three types of categorial links: basic, nested and concurrent, and we show that many domain tasks in hyperlink pipeline, call control and media workflows can be accomplished efficiently by these links. The preliminary experiments indicate the approach is feasible.

In this paper, we present a REST Chart based approach to design and implement RESTful northbound API in SDN. In our approach, the RESTful API is modeled with Petri Net, and REST constraints, such as hypermedia driven, are enforced naturally. The RESTful structure of the designed northbound API can be checked almost automatically based on the described REST Chart model. The proposed approach has been applied to design and implement the northbound API of SDN in data center networks with OpenStack Neutron. To improve the protocol efficiency, we developed a structured caching mechanism for data network applications with SDN, that reduces the response time overheads by more than 65%, while it fully maintains the desired flexibility and scalability of the RESTful northbound API.

Identity resolution is a critical component in any real-time communication system, including WebRTC that relies on Web identities. The difficulty with Web identity resolution is that there is no uniform Web identity scheme and current Web identity schemes decouple Web identities and Web locations. To address this problem for WebRTC without introducing a central authority, we present a secure personal Web identity resolution framework, mirror presence, to dynamically map a user's Web identity to Web locations while the user can move between Web locations. The system is agnostic to Web identity schemes, and it consists of 3 layers of Web protocols and services: 1) bilateral authorization based on OAuth 2.0; 2) Web identity binding based on presence subscription and notification; 3) presence driven call routing and pickup protocols. Each layer of the system can be controlled by users based on security, privacy, and personal preferences. A prototype mirror presence system has been implemented, and experimental results indicated that the approach is feasible and the performance is satisfactory.

Both presence and web service composition are very important to web based real-time communication systems. Although there is some work to combine presence with individual web services, there still lacks an efficient presence composition method to determine presence for composite web services, which is a collection of web services arranged by a process. This paper proposes a distributed computation model, called TAO-Tree (Timed AND-OR tree) to map discrete presence states to continuous time domain such that presence compositions can be carried recursively by arithmetic operations. Furthermore, the paper describes a method to translate the presence delays back to discrete presence states with minimal errors. We implement the TAO-Tree module in a prototype system and the experiments show the approach is feasible and satisfactory.

This paper describes a lightweight service composition mechanism in web browsers, called hyperlink pipeline, to allow users to interactively chain distributed atomic web services into composite services similar to using Unix pipelines. This mechanism addresses the limitation in current Web, which confines users to following predefined hyperlinks and forms. We present the functional composition framework to facilitate recursive hyperlink pipeline construction and execution based on REST services and the HTML5 Micro data model to efficiently implement the framework. The proposed mechanism has been implemented and used in several web applications for real-time communication and collaboration based on WebRTC. The initial tests indicate the approach is feasible and promising.

As REST architectural style gains popularity in the web service community, there is a growing concern and debate on how to design Restful web services (REST API) in a proper way. We attribute this problem to lack of a standard model and language to describe a REST API that respects all the REST constraints. As a result, many web services that claim to be REST API are not hypermedia driven as prescribed by REST. This situation may lead to REST APIs that are not as scalable, extensible, and interoperable as promised by REST. To address this issue, this paper proposes REST Chart as a model and language to design and describe REST API without violating the REST constraints. REST Chart models a REST API as a special type of Colored Petri Net whose topology defines the REST API and whose token markings define the representational state space of user agents using that API. We demonstrate REST Chart with an example REST API. We also show how REST Chart can support efficient content negotiation and reuse hybrid representations to broaden design choices. Furthermore, we argue that the REST constraints, such as hypermedia driven and statelessness, can either be enforced naturally or checked automatically in REST Chart.

Software as a Service (SaaS) has been adopted in a fast pace for applications and services on software clouds. However, the success of SaaS in software cloud cannot obscure the integration challenges faced by developers and enterprise infrastructure IT. Among those challenges, firewall/NAT traversal and security issues often pose a serious bottleneck as enterprises may not be entirely comfortable running mission critical applications outside the corporate firewall. On the other hand, SaaS applications in the cloud need to access enterprise on-premise applications for data exchange and on-premises services. The current approaches through opening special pin-holes on firewall or using dedicated VPNs have encountered a number of limitations and drawbacks. This paper presents a Proxy-based firewall/NAT traversal solution for SaaS integration (PASS). It allows SaaS applications to integrate with on-premise applications without firewall reconfiguration, while maintaining the security of on-premise applications. In addition, this approach is platform and application independent, making the SaaS integration seamless. Moreover, PASS is consistent with the enterprise web browsing infrastructure, and it requires little or no change to enterprise firewall/NAT configurations. In this paper we present the architecture of PASS and address SaaS integration challenges in software cloud, such as security/firewall, performance, and scalability. Experimental study based on our implemented system shows that the proposed approach of PASS is promising to resolve firewall/NAT traversal for SaaS integration with on-premise services.

In recent years, REST (Representational State Transfer) has received much attention for designing scalable web services in various domains. There is an increasing interest on its application to real-time communication web services. Based on a case study of CSTA services, we found that communication services exhibit a combination of complex message exchange and stateful behavior patterns, including multi-states, two-way interaction, event-driven, multi-resources, multi-responses and session management, that need to be modeled properly within the framework of REST. To address these challenges, we propose three REST design patterns: session, event subscription and relationship using REST composition. This approach leads to a systematic one-to-one transformation from CSTA to REST. We hope such a study on a concrete use case can contribute to a better understanding of REST and lead to a seamless convergence of communication services with the infrastructure of the Web.

Software-as-a-Service (SaaS) is an emerging software application delivery model. It has gained an increasing momentum, and it is being adopted at a very fast pace. However, the integration of SaaS applications with enterprise on-premise applications has become a major factor for the success of its adoption. There is a critical need for methods and communication infrastructures that can enable distributed SaaS applications over the data network. This paper presents an approach and a realization of communications enablement of SaaS applications. It focuses on three areas: 1) an approach to enabling communications as a service; 2) a SaaS adaptor framework for integrating SaaS applications with on-premise communications services; and 3) methods and communication middleware for two-way web services crossing enterprise domains. A communication enabled sales opportunity management SaaS application is implemented to verify the proposed approach. Experimental results indicated that the proposed approach is feasible and effective to communication enable SaaS applications in distributed environment that connects both cloud computing and enterprise on-premises systems.

Web service conformance testing checks the correctness of a black box service implementation, and it is the basis of other testings. An efficient formal method for conformance testing is the Chinese postman traversal algorithm that can find minimum-cost test sequences. However, the applicability of this algorithm is in question if data dependences are present in the protocol specification. Also, it suffers from the limited observability problem which is not uncommon in web services. Despite its optimality, the traversal algorithm does not take advantages of the special patterns in web service interfaces observed by developers. To address these issues, we propose an abstract GFSM (guarded finite-state machine) model that unifies and augments the commonly used Moore and Mealy machines with dataflows. Using this abstract model, we formalize the conditions under which the tours with complete data and control coverage are guaranteed, and address the limited observability problem using the equivalence of Moore and Mealy machines. Furthermore, the authors propose a recursive descent traversal algorithm that explores the inverse operation pattern of web services to facilitate incremental development of Web services.

Web services technologies are undergoing some significant changes at several levels, ranging from architectural styles (REST vs.. SOAP) to message formats.To cope with these changes, there is a need to design services in an abstract fashion that is less sensitive to these changes so that our technological and economic investments are protected. To address this issue, this paper proposes an abstract modeling framework, called Infoset, both as a design and analysis tool and a message processing architecture. The benefit of Infoset is demonstrated first on some dual Web services that provide both REST and SOAP access. The Infoset analysis modeling reveals the commonality and differences between dual Web services, making it feasible to design abstract web services. Infoset also leads to a rule based message processing architecture that avoids the problems in current static Object/XML SOAP engines. By dynamically computing the context of Infoset rules in message workflows, dependences between integrated Web services are reduced and the reusability of services logic is increased. A prototype of this architecture is implemented and the preliminary experimental results indicate that the performance is satisfactory.

This paper presents a formal framework for multimodal dialogue systems by applying a set of complexity reduction patterns. The minimalist approach described in this paper combines recursive application of Model-View-Controller (MVC) design patterns with layering and interpretation. It leads to a modular, concise, flexible and dynamic framework building upon a few core constructs. This framework could expedite the development of complex multimodal dialogue systems with sound software development practices and techniques. A XML based prototype multimodal dialogue system that embodies this framework is developed and studied. Experimental results indicate that the proposed framework is effective and well suited for multimodal interaction in complex business transactions.

Web services is a fast moving research field with a profound impact to many critical research areas, ranging from software to communication, from server platforms to mobile endpoints. It emerges as a disruptive technology to various enterprises to deliver services, computing, communication and information to their customers and partners. This talk intends to capture some recent advances of Web services and new Web services applications in software-as-a-services, cloud computing, communication, message centric SOAP engine design, and mobile services computing endpoints. In addition to present the technical perspectives and extension potentials in these new developments, I would like to fill the gap between academic research and industry applications through real examples and use cases, with a goal to demonstrate how these technical advances of Web services are applied to industry products and applications, Standards, servers, clients, and new paradigms in software design and testing.

In this paper, we study the communication over IP based on web services. We describe the service-oriented communication framework of WIP, Web Services Initiation Protocol, and focus on its web centric networking infrastructure. In our approach, WIP endpoints are exposed as resources on the Web, publishable by their UDDI service registries. They can be searched and discovered by regular web search engines, e.g. Google, upon which they can be networked and linked for scalable and distributed communication over IP. In our approach, the WIP endpoint is a web services based communication endpoint for communication over IP. It is also a service endpoint for software-as-a-service (SaaS) that can integrate, marshup, and enable advanced SaaS applications in communication. Moreover, we introduce the presence aware communication framework in WIP utilizing its extensible and service-oriented infrastructure. An advanced WIP communication system prototype has been implemented. It supports the advanced features described in this paper, including distributed communication, networking, security, presence, etc. Experimental results are presented, and the proposed approach and methods are validated.

This paper presents InfoParser, an Infoset driven XML processing mechanism aimed to amend current SOAP engines by substituting InfosetharrXML binding for ObjectharrXML (O/X) mapping, which is responsible for several acute issues in Web services, including O/X impedance mismatch, portability, XML alienation, complexity and memory overhead. The proposed InfoParser approach not only avoids issues in object based framework, but also significantly reduces the complexity in XML centric approach. InfoParser can be described formally by just a few set-theoretic equations, yet it offers a promising alternative to current options in moving Web services towards more semantics driven, light weight and loosely coupled programming paradigm that can work on any devices. We have implemented a prototype InfoParser in Java and the preliminary experiments indicate that the approach is feasible.

In this paper, we describe an approach to discover the control flow graph of Web services for Web services analysis, verification, and testing. For this purpose, three novel methods are proposed. First, we introduce a domain independent RDF Schemas for concise resource oriented functional specification of Web services operations. Secondly, we describe the use of RDF entailment to accurately derive the control flow from the functional specifications. We developed a transformation from RDF graph to SPARQL query to facilitate the RDF entailment which offers flexibility and extensibility over the direct graph matching approach. The third is a linkage based Web services modeling and analysis framework, within which we apply an improved Google PageRank algorithm to efficiently calculate test coverage potential using the derived control flow. We justify that the proposed linkage based Web services modeling and analysis framework is particularly suitable for testing Web services. A prototype of the proposed methods has been implemented and tested on some standard based Web services. Experimental results show that the control flow analysis is quite efficient and accurate, and the coverage based test results of the proposed approach are very promising.

In this paper, we study Web services methods and approaches to enable real-time communication services over IP. This approach extends Web services methodologies from service integration to a service-oriented approach for communication. In particular, we describe the generic Web service based application session management, WS-Session, the two-way full duplex Web service interaction framework, and most importantly, the development of Web service initiation protocol (WIP) which is a full featured Web service and SOA based communication framework for multimedia and voice communication over IP. We show that WIP provides a service oriented architecture which can be extended seamlessly from a P2P endpoint to an endpoint with advanced call control and switching capabilities typically requiring the assistance of a dedicated PBX. A prototype of WIP system is fully developed. Architectural design and system implementation of Web service based communication endpoints are studied and applied to realize service-oriented communication over IP. Advances of WIP indicate the beginning of a full Web service and SOA based communication paradigm that can reshape the converged communication over IP.

This paper describes an approach to discover and verify two fundamental relationships in web service message exchange flow patterns, i.e. support and correlate, based on the abstract WSDL specification of the service operations. Compared to methods used in previous studies, this approach is a significant improvement in that flow chain relations can be discovered and identified based on flexible and inexact match of XML Schemas using differential information in plain WSDL and XML Schema files. This approach is also efficient and extensible as it is based on a tree model of schemas and modular node equivalence rules. Pilot experiments on some web services standards, e.g. WS-Session, WS- Eventing, Parley X, etc., indicated that the proposed method is accurate and effective in identifying the message flow dependencies and related issues in service descriptions. The derived flow pattern relations can be represented as Petri nets that have some practical applications for web service verification, implementation and consumption.

This paper presents an approach for an extensible multimodal interaction dialogue system, R-Flow, based on a recursive application of Model-View-C'ontroller (MVC) design patterns to derive system components and interfaces. This approach leads to a clear separation of three self-contained functional layers in a multimodal dialogue system: modality independent dialog control, synchronization of logical modalities, and physical presentation. These layers are codified and weaved together through standard based XML languages. In particular, it utilizes the standard State-Chart XML (SCXML) for dialog control, SMIL and EMMA based XM-Flow for modality synchronization and interpretation, and a generic XML based binding mechanism to map logical modalities to physical presentations. A prototype system has been implemented for multimodal (e.g. speech, text, and mouse) manipulation of Google map. Our experimental results indicated that such layered and component-based XML MMI system is feasible and the performance of such MMI system is studied and measured.

Service-Oriented Communication (SOC) is a new development in the industry to enable communication through web services and SOA. SOC is to make communication as service and provide a service-oriented architecture to integrate communication in business applications. Recent advances of web service and SOA have made it possible for a full web service and SOA based communication paradigm over IP. This paper is an overview of the recent development in this area with a focus on key technologies that can be applied to web service enablement of communication. In particular, we discuss the generic web service based application session management based on WS-Session, the two-way full duplex web service interaction framework, and the development of Web Service Initiation Protocol (WIP). WIP is a full web service and SOA based communication protocol for multimedia and voice communication over IP. The generic web service approach of WIP overcomes many limitations which would be otherwise difficult to achieve in non-web service based communication methods used today. In addition, we discuss the service composition and orchestration framework based on WS-BPEL, and illustrate the application of this approach through some real use cases.

In this paper, we present WIP - Web service initiation protocol for multimedia and voice communication over IP. WIP is an entirely Web service based communication protocol, consisting of a set of Web service operations for initiating and establishing converged (e.g. multimedia, IM, voice, etc.) communication services over IP. It inherits the principle of separation signaling and media transmission of SIP (session initiation protocol); but it relies on a single Web service stack to provide a full featured communication signaling protocol. WIP opens a new paradigm of Web service based VoIP communication, which is extensible and can be easily integrated in end-to-end SOA solutions. The generic Web service approach used in WIP overcomes many limitations which would be otherwise difficult to achieve in non-Web service based communication methods used today. WIP is based on two-way, full duplex Web service interaction. The communication signaling establishment in WIP is through Web service interactions, and the media negotiation in WIP is modeled as a special Web service "event" subscription, which is fully extensible for various media needs. The signaling messages of WIP are encoded in the standard based SOAP message envelops which can be carried by multiple transport protocols, including HTTP. WIP supports both P2P (peer-to-peer) and B2B (back-to-back) broker mode communication services. A prototype research system has been implemented, and the results indicate that WIP, as a full Web service based communication protocol, is both feasible and advantageous

This paper proposes a conceptual model for modeling static and dynamic event sources by reusing the information in WSDL. A set-theoretic semantics for event subscription is introduced, based on which two metrics, recall and precision, are proposed to measure the accuracy of event subscriptions. We discuss the accuracy of several event subscription strategies under the framework of Web service event subscription standard proposals (WS-Eventing and WS-Notification). Four major types of event broker design patterns are discussed based on two visibility/reachability factors: if sink knows the source and if source can deliver events directly to the sink. The implication on the broker state and message routing is studied in this analysis. A prototype implementation indicated that these design patterns are feasible

In this paper, we present the approach and architecture of TARGET: two-way Web service router gateway, for two-way Web service interaction crossing enterprise domain and firewall. It provides a full support for asynchronous outbound operation and event notification in communication services. TARGET addresses an acute issue for Internet applications that today's enterprise NATs and firewalls only allow outbound HTTP request from the inside to the outside and block any request from the outside to the inside, which is a serious problem for two-way Web services. TARGET is a generic solution to allow two-way Web service interaction to traverse legitimately through NATs and strictly configured firewalls; and it is based on two-way SOAP message tunneling, service local registry, and service routing to bridge two-way Web service interaction. A research TARGET system has been implemented and applied to real time communication services, e.g. conferencing. Extensive experiments on TARGET are performed, and its performance with various sizes of SOAP messages is studied. The applicability and feasibility of TARGET for two-way Web service interaction is verified

This paper presents a synchronization module in multimodal dialogue system architecture based on the model-view-controller (MVC) pattern for human-computer interaction. The MVC pattern is based on a clear separation of objects into three categories, i.e. model for defining and maintaining data, view for rendering interactions based on the data, and controller for coordinating actions and events that affect the model and view(s). As part of our layered multimodal dialog system architecture, this synchronization module in our approach controls the synchronization of multiple modalities, such as speech, mouse and keyboard, by interpreting XML document that incorporates SMIL and EMMA. It isolates dialog model from complex presentations associated with different channels and user interfaces through the adoption of a generic object binding mechanism. These flexibilities lead to enhanced design freedom in multimodal dialog system architecture that supports client based, sever based and distributed solutions

In recent years, various discriminative learning techniques for HMMs have consistently yielded significant benefits in speech recognition. In this paper, we present a novel optimization technique using the minimum classification error (MCE) criterion to optimize the HMM parameters. Unlike maximum mutual information training where an extended Baum-Welch (EBW) algorithm exists to optimize its objective function, for MCE training the original EBW algorithm cannot be directly applied. In this work, we extend the original EBW algorithm and derive a novel method for MCE-based model parameter estimation. Compared with conventional gradient descent methods for MCE learning, the proposed method gives a solid theoretical basis, stable convergence, and it is well suited for the large-scale batch-mode training process essential in large-scale speech recognition and other pattern recognition applications. Evaluation experiments, including model training and speech recognition, are reported on both a small vocabulary task (TI-digits) and a large vocabulary task (WSJ), where the effectiveness of the proposed method is demonstrated. We expect new future applications and success of this novel learning method in general pattern recognition and multimedia processing, in addition to speech and audio processing applications we present in this paper

In this paper, we introduce Web service application session services, WS-session, for Web service based application session management in communication. A two-way Web service framework is described. It supports the use of Web service in the situation of telecommunication where the communication endpoint can be both a client and a peer-to-peer server. We describe the interface design in two-way Web service interaction based on the concept of tightly coupled interface and loosely coupled interface solutions. The proposed approach provides full support of asynchronous outbound operations and event notifications in communication services. It is implemented in a research prototype system that provides both standard based (ECMA-348) services and Avaya service extensions. In addition, we investigated the integration of our proposed approach with BPEL4WS. It allows the user to create new services through a drag-n-drop based service creation environment (ECLIPSE/BPEL4WS). It leads to a paradigm of service creation through Web service composition. More sophisticated telecommunication services are created from composing the basic services.

In this paper, we introduce a theoretical framework for two-way Web service (WS) interaction and interface design based on the concepts of operation reversal and XML type generalization. Under this theoretical framework, two generic types of interface solutions in two-way WS interaction, i.e. tightly coupled (TC) interface and loosely coupled (LC) interface, are formally defined and studied. The proposed theoretical framework is used to design, derive and verify WS interface for full duplex two-way WS interaction. An algorithm of interface verification is described that allows effective WS interface design and verification to support asynchronous two-way WS operation and event notification. The interface verification algorithm in our approach utilizes verification by derivation and verification by content to validate the client interface in two-way WS interaction. Use case studies are performed for both TC and LC interface solutions. A generic infrastructure based on two-way Web service application proxy (2SAP) is described and implemented. The proposed approach is applied to applications of realizing ECMA-348 for WS enablement of telecommunication.

In this paper, we present the theoretical framework of minimum classification error (MCE) training of generalized linear classifiers for text classification. We show that many important text classifiers, either probabilistic or non-probabilistic, can be unified under this framework, and the proposed MCE classifier training approach can be applied to improve the classifier performance. In addition, we describe an effective MCE classifier training algorithm that uses AdaBoost to generate alternative initial classifiers, as opposed to combining multiple classifiers as it is typically used. This method is applied to MCE classifier training to overcome local minimums in optimal classifier parameter search, utilizing the fact that the family of generalized linear classifiers is closed under AdaBoost. Moreover, we extend the loss function in MCE training to incorporate training sample prior distributions to compensate the imbalanced training data distribution in each category. Experimental studies are performed on the text classification tasks, and the significant classification error reductions of 25% - 55% are observed.

We present an approach for converged communication services over IP, based on the concept of WSIP (Web service SIP). In our approach, each WSIP node is both a SIP endpoint that communicates in the SIP world through SIP signaling, and a Web service SOAP node that provides a native and generic service integration environment for binding SIP based communication in Web services. The unification of SIP with Web service provides a new converged communication paradigm. It allows dynamic service discovery and binding that integrate SIP as a component in the business transaction. The clear separation of service integration environment and SIP signaling in our approach maintains the simplicity and efficiency of the existing SIP protocol, while leverages the power of both methods in IP based communication services. The ubiquitous service integration nature of Web service provides a disciplined solution to maintain and update large number of SIP endpoints from different installation bases. It enables dynamic monitoring, repairing, and updating SIP UA endpoints, which is essential to the success of VoIP using SIP. The proposed approach was implemented in a prototype research system, and several advantages of WSIP over the existing approaches are observed.

In this paper, we study the voice portal services in packet network and VoIP environment. An extensible VoIPTeleserver for VoIP in SIP (Session Initiation Protocol) environment is described. It is based on the concept of dialogue system and web convergence that separates the channel dependent media resources from the application creation environment. It supports XML based service applications for multiple channels including voice, DTMF, IM and chat over IP. Special attention is given to the adverse effect of delay, jitter and packet loss for voice portal services over IP. In particular, case studies of DTMF service in voice portal under adverse channel conditions are performed. The compounding effects of multiple channel impairments to DTMF in voice portal services over IP are revealed. The potential high error rate indicates that the data redundancy method as proposed in RFC 2198 is needed for DTMF in order to achieve reliable voice portal services over IP.

This paper presents the software architecture for a new paradigm of wireless Web and dialogue system convergence and its corresponding approach to deliver multimodal multimedia enterprise communication services in the wireless environment. It uses an event-triggered service delivery method that separates service interaction and wireless web-based service delivery. A customized content presentation approach is adopted to transform the content of the service into a format suitable for presentation on the target devices. The system also contains a service delivery monitoring and backup mechanism, which is capable of sending the service content using other communication channels according to the user's preference and the channel conditions. A prototype system was implemented based on the proposed approach, which demonstrates the performance advantage of our method in the enterprise service application.

In this paper, a concatenated "super" string model based minimum classification error (MCE) model adaptation approach is described. We show that the error rate minimization in the proposed approach can be formulated into maximizing a special ratio of two positive functions. The proposed string model is used to derive the growth transform based error rate minimization for MCE linear regression (MCELR). It provides an effective solution to apply MCE approach to acoustic model adaptation with sparse data. The proposed MCELR approach is studied and compared with the maximum likelihood linear regression (MLLR) based model adaptation. Experiments on large vocabulary speech recognition tasks are performed. Experimental results indicate that the proposed MCELR model adaptation can lead to significant speech recognition performance improvement and its performance advantage over the MLLR based approach is observed even when the amount of adaptation data is sparse.

In this paper, we present an approach of integrating SIP (session initiation protocol) in converged multimodal/multimedia communication services. An architecture of multimedia technology integration platform (MTIP) in converged communication environment with SIP is described. This distributed dialogue system infrastructure is based on the concept of dialogue system and web convergence, where applications and communication services can be enabled through task specific markup languages based on XML. The loosely coupled open architecture in our approach is highly extensible. It allows an easy extension to include SIP in converged multimodal/multimedia communication services. The concept and structure of VoIPTeleServer is described, which interfaces to the VoIP world and works as a broker between the SIP environment and MTIP to deliver converged communication services. A prototype of VoIPTeleServer was implemented, and services and applications based on SIP and MTIP convergence are constructed.

In this paper, a concatenated "super" string model based minimum classification error (MCE) model adaptation approach is described. We show that the error rate minimization in the proposed approach can be formulated into maximizing a special ratio of two positive functions. The proposed string model is used to derive the growth transform based error rate minimization for MCE linear regression (MCELR). It provides an effective solution to apply MCE approach to acoustic model adaptation with sparse data. The proposed MCELR approach is studied and compared with the maximum likelihood linear regression (MLLR) based model adaptation. Experiments on large vocabulary speech recognition tasks are performed. Experimental results indicate that the proposed MCELR model adaptation can lead to significant speech recognition performance improvement and its performance advantage over the MLLR based approach is observed even when the amount of adaptation data is sparse.

In this paper, the theoretical framework of maximum a posteriori linear regression (MAPLR) based variance adaptation for continuous density HMMs is described. In our approach, a class of informative prior distribution for MAPLR based variance adaptation is identified, from which the close form solution of MAPLR based variance adaptation is obtained under its EM formulation. Effects of the proposed prior distribution in MAPLR based variance adaptation are characterized and compared with conventional maximum likelihood linear regression (MLLR) based variance adaptation. These findings provide a consistent Bayesian theoretical framework to incorporate prior knowledge in linear regression based variance adaptation. Experiments on large vocabulary speech recognition tasks were performed. The experimental results indicate that significant performance gain over the MLLR based variance adaptation can be obtained based on the proposed approach.

In this paper, a framework of minimum classification error (MCE) model adaptation for continuous density HMMs is proposed based on the approach of "super " string model. We show that the error rate minimization in the proposed approach can be formulated into maximizing a special ratio of two positive functions, and from that a general growth transform algorithm is derived for MCE based model adaptation. This algorithm departs from the generalized probability descent (GPD) algorithm, and it is well suited for model adaptation with a small amount of training data. The proposed approach is applied to linear regression based variance adaptation, and the close form solution for variance adaptation using MCE linear regression (MCELR) is derived. The MCELR approach is evaluated on large vocabulary speech recognition tasks. The relative performance gain is more than doubled on the standard (WSJ Spoke 3) database, comparing to maximum likelihood linear regression (MLLR) based variance adaptation for the same amount of adaptation data. 1.

word clusters in natural language call routing (NLCR) is proposed. This approach utilizes an automatic word class clustering algorithm to generate word classes from the word based training corpus. In our approach, the information gain (IG) based term selection is used to combine both word term and word class information in NLCR. A joint latent semantic indexing natural language understanding algorithm is derived and studied in NLCR tasks. Comparing with word term based approach, an average performance gain of 10.7% to 14.5% is observed averaged over various training and testing conditions.

We present an approach for secure delivery of multimodal services to end users over heterogeneous converged networks. Our approach leads to an easily adaptable and extensible system architecture, in which system components interact with each other through loosely coupled interfaces based on asynchronous exchange of messages. It also effectively separates service logic from service delivery, through the introduction of a secure broker between the end user devices and the enterprise level application that contains the service logic. This separation also provides additional levels of security at the application level. This paper describes how our approach provides the features mentioned above. We have also validated our approach through the implementation of a prototype system, a description of which is also provided.

This paper presents an approach based on wireless Web and dialogue system convergence for multimodal, multimedia communication services over converged networks. It uses a unique event-triggered service delivery method that separates service interaction and wirelessWeb-based service delivery. Our approach results in an architecture of WAP-based, location-independent, wireless service monitoring, notification and service access for mobile users that can be applied over various network bearers in enterprise environment. A new wireless secure server (WSS) provides secure connections to dialogue systems. It allows one-key switching to retrieve the multimedia content. A prototype system was implemented and performance advantages were demonstrated on several enterprise service applications.

In this paper, we describe an approach of using a discriminative term selection process based on information grain (IG) to improve the performance of the latent semantic indexing (LSI).

In this paper, we describe a distributed multimodal dialogue system architecture based on the concept of hybrid-VoiceXML. It utilizes a special hybrid-construct to integrate multiple multimedia, multimodal processes into one dialogue that includes VoiceXML as its voice modality. The hybridconstruct in our approach has several important functions. It provides an additional abstraction layer for dynamic dialogue generation, which can greatly improve the efficiency and flexibility of the dialogue system. Under the proposed approach, the dialogue control between each interaction channel can be exchanged through the interface of a dynamic XML page. Several case studies are performed. It indicates that the proposed hybrid-VoiceXML approach is highly extensible. It can be used to form platform independent and distributed extensions for multimodal dialogue interaction beyond voice.

In this paper, an approach for robust singing signal detection in speech/music discrimination is proposed and applied to applications of audio indexing. Conventional approaches in speech/music discrimination can provide reasonable performance with regular music signals but often perform poorly with singing segments. This is due mainly to the fact that speech and singing signals are extremely close and traditional features used in speech recognition do not provide a reliable cue for speech and singing signal discrimination. In order to improve the robustness of speech/music discrimination, a new set of features derived from the harmonic coefficient and its 4 Hz modulation values are developed in this paper, and these new features provide additional and reliable cues to separate speech from singing. In addition, a rule-based post-filtering scheme is also described which leads to further improvements in speech/music discrimination. Source-independent audio indexing experiments on the PBS Skills database indicate that the proposed approach can greatly reduce the classification error rate on singing segments in the audio stream. Comparing with existing approaches, the overall segmentation error rate is reduced by more than 30%, averaged over all shows in the database.

ABSTRACT In this paper, a dialogue system for natural language based call steering is described and studied. The system is based on natural language speech recognition and understanding within a mixed initiative dialogue. The system is implemented on Bell Labs. Speech Technology Integration Platform (BLSTIP) using dialogue and natural language understanding components from BT laboratories. A prototype system in the operator service domain [2] is described. In order to improve the acoustic and language modeling for natural language based dialogue applications, various approaches are described and studied. The structure of the dialogue manager is also presented in which mixed-initiative dialogue can be supported with efficiency. Call classification and steering experiments were performed. The results confirm the efficacy of the proposed approach. 1. INTRODUCTION Natural language dialogue between human and machine is a challenge. In order to make a natural language based dialogue system successful, various efforts are made to improve the accuracy, flexibility and robustness of the system component technologies, such as speech recognition, speech understanding, dialogue generation and dialogue manager, text-to-speech synthesis, etc. Such a complex dialogue application imposes stringent requirements on the flexibility of the system platform. One of the drawbacks in systems deployed in the past is the limitation imposed by the finite state grammar on the language that a user can use to communicate with the machine. Although such constraint alleviates the complexity and problem in recognizing human speech, it becomes an obstacle to support more powerful, user friendly and flexible dialogue systems for mixed-initiative dialogues. In this paper, we study issues encountered in designing and implementing a natural language based call steering application for telephone service calls. This is a complicated application, and it performs a detailed diagnostic dialogue to identify the service problem, such as a troubled telephone line and etc., that the user is experiencing. It provides the desired service after receiving user’s consent and confirmation [2]. In the prototype system studied in this paper, the dialogue can go deep through many turns. The natural language based request and query from the user is recognized through natural language based automatic speech recognition. There is no constraint on the way that the user should communicate to the system. It allows the user to make direct requests as well as provide a description of the problem where the final action will be identified as the outcome of the dialogue. A call classifier provides natural language understanding based on the word string from the speech recognition output. The dialogue manager uses this understanding to determine the next appropriate system action. The organization of this paper is as follows. In Section 2, the dialogue system architecture and design are presented which support natural language based mixed-initiative dialogue applications such as call steering, movie locator, etc. Section 3 is devoted to natural language based speech recognition and statistical language modeling for dialogue applications. Section 4 is concentrated on the dialogue manager design and automatic query generation. Call classification and steering are studied in Section 5 and results are given based on a case study in a telephone service application.

A discriminant function-based minimum recognition error rate pattern recognition approach is described and studied for various applications in speech processing. This approach departs from the conventional paradigm, which links a classification/recognition task to the problem of distribution estimation. Instead, it takes a discriminant function based statistical pattern recognition approach. The suitability of this approach for classification error rate minimization is established through a special loss function. It is meaningful even when the model correctness assumption is known to be not valid. We study the theoretical basis of this approach and compare it with various criteria used in speech recognition. We differentiate the method of classifier design by way of distribution estimation and the discriminant function methods of minimizing classification error rate, based on the fact that in many realistic applications, such as speech recognition, the true distribution form of the source is rarely known precisely, and without model correctness assumption, the classical optimality theory of the distribution estimation approach cannot be applied directly. We discuss issues in this new classifier design paradigm and present various extensions of this approach to classifier design applications in speech processing.

Methods of improving the robustness and accuracy of acoustic modeling using decision tree based state tying are described. A new two-level segmental clustering approach is devised which combines the decision tree based state tying with agglomerative clustering of rare acoustic phonetic events. In addition, a unified maximum likelihood framework for incorporating both phonetic and nonphonetic features in decision tree based state tying is presented. In contrast to other heuristic data separation methods, which often lead to training data depletion, a tagging scheme is used to attach various features of interest and the selection of these features in the decision tree is data driven. Finally, two methods of using multiple-mixture parameterization to improve the quality of the evaluation function in decision tree state tying are described. One method is based on the approach of k-means fitting and the other method is based on a novel use of a local multilevel optimal subtree. Both methods provide more accurate likelihood evaluation in decision tree clustering and are consistent with the structure of the decision tree. Experimental results on Wall Street Journal corpora demonstrate that the proposed approaches lead to a significant improvement in model quality and recognition performance.

In this paper, an approach of the penalized Bayesian information criterion (pBIC) for decision tree state tying is described. The pBIC is applied to two important applications. First, it is used as a decision tree growing criterion in place of the conventional approach of using a heuristic constant threshold. It is found that original BIC penalty is too low and will not lead to a compact decision tree state tying model. Based on Wolfe's modification to the asymptotic null distribution, it is derived that two times BIC penalty should be used for decision tree state tying based on pBIC. Secondly, pBIC is studied as a model compression criterion for decision tree state tying based acoustic modeling. Experimental results on a large vocabulary (Wall Street Journal) speech recognition task indicate that a compact decision tree could be achieved with almost no loss of the speech recognition performance.

A unified maximum likelihood framework of incorporating phonetic and non-phonetic features in decision tree based acoustic modeling is proposed. Unlike phonetic features, non-phonetic features in this context are those features, which cannot be derived from the phoneme identities. Although non-phonetic features are used in speech recognition, they are often treated separately and based on various heuristics. In our approach, non-phonetic features are included as additional tags to the decision tree clustering. Moreover, the proposed tagged decision tree is based on the full training data, and therefore, it alleviates the problem of training data depletion in building specific feature dependent acoustic models. Experimental results indicate that up to 10% word error rate reduction can be achieved in a large vocabulary (Wall Street Journal) speech recognition task based on the proposed approach.

A fast segmental clustering approach to decision tree tying based acoustic modeling is proposed for large vocabulary speech recognition. It is based on a two level clustering scheme for robust decision tree state clustering. This approach extends the conventional segmental K-means approach to phonetic decision tree state tying based acoustic modeling. It achieves high recognition performances while reducing the model training time from days to hours comparing to the approaches based on Baum-Welch training. Experimental results on standard Resource Management and Wall Street Journal tasks are presented which demonstrate the robustness and efficacy of this approach.

In this paper we discuss the role of language modeling in a novel natural language dialogue system designed to automatically route incoming customer calls. We arrive at two significant conclusions: First, standard word error rate measures do not reflect application specific requirements; highly reliable content extraction is possible with relatively high word error rates. Secondly blending human-human data with human-machine data did not improve the performance in language modeling. 1.

We propose a signal prototype classification and evaluation framework in acoustic modeling. Based on this framework, a new tree-structured likelihood function is derived. It uses a designated cluster kernel f/sub m//sup C/ for signal prototype classification and a designated cluster kernel f/sub m//sup L/ for likelihood evaluation of outlier or tail events of the cluster. A minimum classification error (MCE) rate training approach is described for designing tree-structured likelihood function. Experimental results indicate that the new tree-structured likelihood function significantly improves the acoustic resolution of the model. It has a more significant speedup in decoding than the one obtained from the conventional approach.

In this paper, an approach to continuous speech recognition based on a layered self-adjusting decoding graph is described. It utilizes a scaffolding layer to support fast network expansion and releasing. A two level hashing structure is also described. It introduces self-adjusting capability in dynamic decoding on a general re-entrant decoding network. In stack decoding, the scaffolding layer in the proposed approach enables the decoder to look several layers into the future so that long span inter-word context dependency can be exactly preserved. Experimental results indicate that highly efficient decoding can be achieved with a significant savings on recognition resources.

We present a generic speech technology integration platform for application development and research across different domains. The goal of the design is two-fold: On the application development side, the system provides an intuitive developer’s interface defined by a high level application definition language and a set of convenient speech application building tools. It allows a novice developer to rapidly deploy and modify a spoken language dialogue application. On the system research and development side, the system uses a thin, ‘broker’ layer to separate the system application programming interface from the service provider interface. It makes the system easy to incorporate new technologies and new functional components. We also use a domain independent acoustic model set to cover US English phones for general speech applications. The system grammar and lexicon engine creates grammars and lexicon dictionaries on the fly to enable a practically unrestricted vocabulary for many recognition and synthesis applications.

Unlike other image templates, visemes have identities in two different media. In audio domain, they are often related to basic linguistic units such as phonemes. In image domain, they are defined by the images of human articulators, such as mouth shapes, chin movements, etc. In this paper, an approach of extracting visemes from both image and acoustic domains is presented. In image domain, the mouth shapes, represented by feature points on inner lip contours, are extracted through face tracking and mouth image analysis. In acoustic domain, viseme segments are obtained automatically by aligning phoneme strings to audio signals through a Viterbi alignment process.

This paper focuses on one aspect of achieving flexible speech recognition, namely, improving the ability to cope with naturally spoken utterances through discriminative utterance verification. We propose an algorithm for training utterance verification systems based on the minimum verification error (MVE) training framework. Experimental results on speaker-independent connected digits, show a significant improvement in verification accuracy when the discriminant function used in MVE training is made consistent with the confidence measure used in utterance verification. At a 10% rejection rate, for example, the new proposed method reduces the string error rate by a further 22.7% over our previously reported results in which the MVE based discriminative training was not incorporated.

In this paper, a wave decoder based on the general re-entrant network for continuous speech recognition is described.The decoder design is based on the concept of self-adjusting decoding graph in which the decoding network is expanded and released frame-synchronously.The fast network expansion and release are made possible by utilizing a novel dynamic network scaolding layer.The self-adjusting decoding graph is obtained by slicing the traditional decoding network horizontally for separation of dierent knowledge sources and vertically according to each time instant in search.A two l a y er hashing structure and an admissible arc predication scheme are described.These methods signicantly reduce the arc mortality rate, a problem which plagues the eciency of the dynamic decoder.Experimental results demonstrate that an order of magnitudereduction of decoding resources can be achieved based on the proposed approach.

Acoustic mismatch encountered in various training and testing conditions of hidden Markov model (HMM) based systems often causes severe degradation in speech recognition performance. For telephone based speech recognition tasks, acoustic mismatch can arise from various sources, such as variations in telephone handsets, ambient noise, and channel distortions. This paper presents three techniques for blind channel equalization, namely, cepstral mean subtraction (CMS), signal bias removal (SBR) and hierarchical signal bias removal (HSBR). Experimental results on various connected digits databases show a reduction in the digit error rate by 16%, 21%, and 28% when employing CMS, SBR, and HSBR, respectively. Our results also demonstrate that the HSBR technique outperforms SBR and CMS on every sub-data collection and exhibits consistent improvements even for short utterances.

We discuss some issues related to acoustic assisted image coding and animation. An approach of talker independent acoustic assisted image coding and animation scheme is studied. A perceptually based sliding window encoder is proposed. It utilizes the high rate (or oversampled) viseme sequence from the audio domain for image domain viseme interpolation and smoothing. The image domain visemes in our approach are dynamically constructed from a set of basic visemes. The look-ahead and look-back moving interpolations in the proposed approach provide an effective way to compensate the mismatch between auditory and visual perceptions.

We utilize speech information to improve the quality of audio-visual communications such as video telephony and videoconferencing. We show that the marriage of speech analysis and image processing can solve problems related to lip synchronization. We present a technique called speech-assisted frame-rate conversion, and apply it to coding of talking head video. Demonstration sequences are presented. Extensions and other applications are outlined.

We propose a new accurate string hypothesization algorithm to find the N-best multiple string hypotheses in continuous speech recognition. The algorithm differs from the conventional N-best search algorithms in that it allows the use of the same set of long term language model scores and the detailed context-dependent subword models such as inter-word context dependent triphone models in both forward and backward search for high performance speech recognition. It is an extension of the tree-trellis N-best search algorithm[1]. The inter-word context dependency is exactly preserved in both forward partial path map preparation and the proposed backward N-best multiple string hypothesis tree search. The search efficiency is maximized by applying the same high resolution acoustic and language models in both search directions. When search heuristics are used, the proposed approach provides a more accurate string model matching than that of the conventional frame-synchronous Viterbi beam search decoder.>

In this paper, a minimum error rate pattern recognition approach to speech recognition is studied with particular emphasis on the speech recognizer designs based on hidden Markov models (HMMs) and Viterbi decoding. This approach differs from the traditional maximum likelihood based approach in that the objective of the recognition error rate minimization is established through a specially designed loss function, and is not based on the assumptions made about the speech generation process. Various theoretical and practical issues concerning this minimum error rate pattern recognition approach in speech recognition are investigated. The formulation and the algorithmic structures of several minimum error rate training algorithms for an HMM-based speech recognizer are discussed. The tree-trellis based N-best decoding method and a robust speech recognition scheme based on the combined string models are described. This approach can be applied to large vocabulary, continuous speech recognition tasks and to speech recognizers using word or subword based speech recognition units. Various experimental results have shown that significant error rate reduction can be achieved through the proposed approach.

A novel training algorithm, segmental GPD (generalized probabilistic descent) training, for a hidden Markov model (HMM)-based speech recognizer using Viterbi decoding is proposed. This algorithm is based on the principle of minimum recognition error rate in which segmentation and discriminative training are jointly optimized. Various issues related to the special structure of HMM in segmental GPD training are studied. The authors tested this algorithm on two speaker-independent recognition tasks. The first experiment involves English E-set. Segmental GPD training was directly applied to HMM generated from nonoptimal uniform segmentation. A recognition rate of 88.7% was achieved on English E-set with whole word HMM. The second experiment involves the connected digits TI-database. Segmental GPD training was applied to HMM which were already trained using conventional training methods. A string recognition rate of 98.8% was achieved on 10-state word based HMM through segmental GPD training.>

A modulo sigma-delta modulator is introduced, and the behavior of the quantization error is derived. The system consists of a modulo limiter followed by a sigma-delta modulator. The limiter confines the input to the no-overload region of the sigma-delta modulator, and the modulo arithmetic performed by the limiter is amenable to recently developed techniques for the exact analysis of quantizer error behavior in sigma-modulators with bounded inputs. The quantization error behavior is derived for a modulator driven by a quasi-stationary random process. The limit distribution and the power of the quantization error are found. Except for some singular cases, the normalized quantization error is uniformly distributed in (-1/2, 1/2). The power spectrum and the autocorrelation function of the quantization error with a causal ARMA (p, q) process input are also derived. It is shown that the quantization noise is white when the input is a random process with stationary independent increments. Simulation results support the theoretical analysis.>

Dithering inside the loop of sigma-delta and multistage sigma-delta modulators is studied. It is shown under certain conditions that dithering inside the loop of the sigma-delta modulator has the effect of attenuating the discrete quantization error spikes. It does not make the quantization error white. For a multistage sigma-delta modulation system, inside-the-loop dithering makes the quantization error white for a system with three stages. Simulation results are presented to support the theoretical analysis.>

An exact analysis of the quantization error of the single-loop single-stage sigma-delta modulator with a multilevel quantizer and single-bit feedback is given. It is shown that the output sequence of this system is identical to that of a coder using a multilevel digital-to-analog converter (DAC) in the feedback loop provided that an extra bit is available in the forward quantizer.>

The spectrum of the quantization error in a dithered sigma-delta modulator is derived under the constraint that the dithering signal does not cause overload. The results apply to DC, sinusoidal, and more general quasi-stationary signals. It is shown in the case of a simple sigma-delta modulation that no-overload dithering can smooth the error spectrum and can make the quantization error asymptotically uncorrelated with the input. It does not, however, make the quantization error white. In the case of multistage sigma-delta modulation with the appropriate dithering, the quantization error becomes white, even for a system with only two stages. The signal-to-quantization-noise ratio (SQNR) is derived for sigma-delta and multistage sigma-delta oversampled analog-to-digital conversion with additive dithering. Simulation results, are presented to support the theoretical analysis.>

A time-domain analysis of sigma-delta modulation with linear filter decoding has been provided. The decoding operation of the 1-b output from a sigma-delta modulator can always be decomposed, and the final output of the decoding filter is the sum of signal term s/sub H/ and noise term n/sub H/. The exact form of s/sub H/ and n/sub H/ has been derived. Two typical sinc and since/sup 2/ digital decoding filters are studied. The only condition imposed on the input x/sub n/ is that x/sub n/ belongs to (-b,b), to avoid overloading the sigma-delta modulator. The results apply even to the case when the signal is not oversampled.>

An exact nonlinear difference equation is derived and solved for a simple sigma-delta modulator consisting of a discrete-time integrator and a binary quantizer inside a single feedback loop and an arbitrary input signal. It is shown that the system can be represented as an affine operation (discrete-time integration of a biased input) followed by a memoryless nonlinearity. An extension of the transform method for the analysis of nonlinear systems is applied to obtain formulas for first- and second-order time-average moments of the binary quantization noise, including the sample mean, energy, and autocorrelation. The results are applied to the special case of a sinusoidal input signal to evaluate these time averages and the power spectrum. In the limit of large oversampling ratios, the marginal moments behave as if the quantization noise had a uniform distribution. The spectrum is neither white nor continuous, however, even in the limit of large oversampling ratios.>

A theoretical basis is provided for multistage sigma-delta modulation (MSM), which is a cascade realization of several single-loop sigma-delta modulators with a linear combinatorial network. Equations are derived describing the output and the quantization noise of MSM for an arbitrary input signal, and the noise-shaping characteristic of MSM is investigated. The spectral characteristics of an m-stage sigma-delta modulator with both DC and sinusoidal inputs are developed. For both types of inputs the binary quantizer noise of the mth (m>or=3) quantizer, which appears at the output as an mth order difference, is asymptotically white, uniformly distributed, and uncorrelated with the input level. It is also found that for an m-stage sigma-delta quantizer with either an ideal low-pass filter or a sinc/sup m+1/ filter decoder, the average quantization noise of the system is inversely proportional to the (2m+1)th power of the oversampling ratio. This implies that the high-order systems are favourable in terms of the trade-off between the quantization noise and oversampling ratio. Simulation results are presented to support the theoretical analysis.>