In the commercial development of a stream processing application, it’s frequently required to connect to external existing services or applications from it in order to, for example, retrieve data from a data store server. If we connect them through message queues (MQs) adopting exactly-once (EO) message delivery semantics to avoid duplicate processing of the same messages caused by a failure recovery of the upstream backup technique that involves rereading of messages stored in MQs, it causes serious performance degradation on the inquiries. This becomes a major obstacle in realizing real-time applications.This paper investigates the kind of problems that practically happen in failure recovery of a stream processing application that inquires to external services or applications through MQs adopting at-least-once (ALO) message delivery semantics instead of EO to avoid the latency overhead. The efforts revealed three typical consistency problems on message sequencing that happen in failure recovery. By solving each of the three problems, our developed R3CM technique to inquire to external services/applications from a stream processing application turns the adoption of ALO semantics in MQs into an attractive option to avoid latency overhead.

Nowadays, structured data such as sales and business forms are stored in data warehouses for decision makers to use. Further, unstructured data such as emails, HTML texts, images, videos, and office documents are increasingly accumulated in personal computer storage due to spread of mailing, WWW, and word processing. Such unstructured data, or what we call multimedia documents, are larger in volume than structured data and precious as corporate assets as well. So we need a document warehouse as a software framework where multimedia documents are analyzed and managed for corporate-wide information sharing and reuse like a data warehouse for structured data. We describe a prototype document warehouse system, which supports management of simple and compound documents, keyword-based and content-based retrieval, rule-based classification, SOM-based clustering, and XML data query and view rules.

We propose a parallel directory structure, Fat-Btree, to improve high speed access for parallel database systems in shared nothing environments. The Fat-Btree has a threefold aim: to provide an indexing mechanism for fast retrieval in each processor; to balance the amount of data among distributed disks, and to reduce synchronization costs between processors during update operations. We use a probability based model to compare the throughput and response time of the Fat-Btree with two ordinary parallel Btree structures, with copies of a whole Btree in each processor and storing index nodes in a processor. The comparison results indicate that the Fat-Btree is suitable for actual parallel database systems that accept update operations.