ISIS (Intelligent Speech for Information Systems) is a trilingual spoken dialog system (SDS) for the stocks domain. It handles two dialects of Chinese (Cantonese and Putonghua) as well as English---the predominant languages in our region. The system supports spoken language queries regarding stock market information and simulated personal portfolios. The conversational interface is augmented with a screen display that can capture mouse-clicks as well as textual input by typing or stylus-writing. Real-time information is retrieved directly from a dedicated Reuters satellite feed. ISIS provides a system test-bed for our work in multilingual speech recognition and generation, speaker authentication, language understanding and dialog modeling. This article reports on our new explorations within the context of ISIS, including: (i) adaptivity to knowledge scope expansion; (ii) asynchronous human-computer interaction by task delegation to software agents; (iii) multi-threaded online interaction and offline delegation dialogs with interruptions for task switching.

This paper presents our approach towards natural language response generation for mixed-initiative dialogs in the CUHK Restaurants domain. Our experimental corpus consists of about 4000 customer requests and waiter responses. Every request/response utterance is annotated with its task goal (TG) and dialog act (DA). The variable pair {TG, DA} is used to represent the dialog state. Our approach involves a set of corpus-derived dialog state transition rules in the form of {TG, DA}request ∆ {TG, DA}response. These rules encode the communication goal(s) and initiatives of the request/ response. Another set of hand-designed rules associate each response dialog state with one or more text generation templates. Upon testing, our system parses the input customer request for concept categories and from these infers the TG and DA using trained Belief Networks. Application of the dialog state transition rules and text generation templates automatically generates a (virtual) waiter response. Ten subjects were invited to interact with the system. Performance evaluation based on Grice's maxims gave a mean score of 4 on a five-point Likert scale and a task completion rate of at least 90%.

This paper presents a progress update of our ISIS 1 trilingual spoken dialog system. As described in [8], this is a conversational system for the stocks domain, and supports interactions in the languages of our region – English and two dialects of Chinese (Mandarin and Cantonese). ISIS provides a system test-bed for our initial explorations with the CORBA architecture, and delegation to KQML (Knowledge Query and Manipulation Language) agents. CORBA offers the advantages of interoperability, scalability and location transparency in client/server systems development. Users can delegate tasks to software agents to help monitor information (e.g. a drop in the price of a pre-specified stock), and generate user alert messages. Our current work presents new research directions in the context of ISIS: (i) automatic incorporation of newly listed stocks into our system’s knowledge base; (ii) switching between on-line interaction and off-line delegation in a single dialog thread. We will also report on enhancements in the system’s architecture and features (e.g. automatic end-point detection). delegation in a single dialog thread. We will also report on enhancements in the system’s architecture and features (e.g. automatic end-point detection). 2. System Architecture Previous work in the development of software infrastructures for dialog systems include [11] 2 [2]. Over the past year, we have continued to explore the development of a spoken dialog system based on the CORBA architecture. This middleware resides in between the operating system and the application layer 1.

ISIS, which abbreviates Intelligent Speech for Information Systems, is a trilingual spoken dialog system (SDS) for the financial domain. It handles two dialects of Chinese (Cantonese and Putonghua), as well as English the predominant languages in our region. The system supports spoken language queries regarding stock market information and simulated personal portfolios. Real-time information is retrieved directly from a dedicated Reuters satellite feed. ISIS provides a system test-bed for our work in multilingual speech recognition and generation, speaker authentication, language understanding and dialog modeling. Furthermore, ISIS supports our initial explorations in: (i) CORBA's interoperability and scalability for SDS development; in conjunction with (ii) asynchronous human-computer interaction by delegation to KQML software agents...