The XNET XML content network was designed to implement efficient and reliable distribution of structured XML content to very large populations of consumers. For that purpose, our system integrates several technologies: the routing protocol XROUTE makes extensive use of subscription aggregation to limit the size of routing tables while ensuring perfect routing (that is, an event is forwarded to a link only if it leads to an interested consumer). The filtering engine XTRIE uses a sophisticated algorithm to match incoming XML documents against large populations of tree-structured subscriptions, whereas the XSEARCH subscription management algorithm enables the system to efficiently manage large and highly dynamic consumer populations. Finally, our XNET system integrates reliability mechanisms to guarantee that its state is consistent with the consumer population and implements several approaches to fault tolerance to recover from various types of router and link failures. We have analyzed the efficiency of our techniques with various simulations, and to assess the performance of our system in realistic settings and show that it is perfectly suitable for large-scale distributed environments, we have performed a large-scale experimental deployment on the PlanetLab testbed.

With the advent of XML as the de facto language for data publishing and exchange, scalable distribution of XML data to large, dynamic populations of consumers remains an important challenge. Content-based publish/subscribe systems offer a convenient design paradigm, as most of the complexity related to addressing and routing is encapsulated within the network infrastructure. To indicate the type of content that they are interested in, data consumers typically specify their subscriptions using a tree-pattern specification language (an important subset of XPath), while producers publish XML content without prior knowledge of any potential recipients. Discovering semantic communities of consumers with similar interests is an important requirement for scalable content-based systems: such "semantic clusters" of consumers play a critical role in the design of effective content-routing protocols and architectures. The fundamental problem underlying the discovery of such semantic communities lay in effectively evaluating the similarity of different tree-pattern subscriptions based on the observed document stream. In this paper, we propose a general framework and algorithmic tools for estimating different tree-pattern similarity metrics over continuous streams of XML documents. In a nutshell, our approach relies on continuously maintaining a novel, concise synopsis structure over the observed document stream that allows us to accurately estimate the fraction of documents satisfying various Boolean combinations of different tree-pattern subscriptions. To effectively capture different branching and correlation patterns within a limited amount of space, our techniques use ideas from hash-based sampling in a novel manner that exploits the hierarchical structure of our document synopsis. Experimental results with various XML data streams verify the effectiveness of our approach.

Content-based publish/subscribe systems are usually implemented as a network of brokers that collaboratively route messages from information providers to consumers. A major challenge of such middleware infrastructures is their reliability and their ability to cope with failures in the system. In this paper, we present the architecture of the XNET XML content network and we detail the mechanisms that we implemented to gracefully handle failures and maintain the system state consistent with the consumer population at all times. In particular, we propose several approaches to fault tolerance so that our system can recover from various types of router and link failures. We analyze the efficiency of our techniques in a large scale experimental deployment on the PlanetLab testbed. We show that XNET does not only offer good performance and scalability with large consumer populations under normal operation, but can also quickly recover from system failures.

In content networks, messages are routed on the basis of their content and the interests (subscriptions) of the message consumers. This form of routing offers an interesting alternative to unicast or multicast communication in loosely-coupled distributed systems with large number of consumers, with diverse interests, wide geographical dispersion, and heterogeneous resources (e.g., CPU, bandwidth). In this paper, we propose a novel protocol for content-based routing in overlay networks. This protocol guarantees perfect routing (i.e., a message is received by all, and only those, consumers that have registered a matching subscription) and optimizes the usage of the network bandwidth. Furthermore, our protocol takes advantage of subscription aggregation to dramatically reduce the size of the routing tables, and it fully supports dynamic subscription registrations and cancellations without impacting the routing accuracy. We have implemented this protocol in the application-level routers of an overlay network to build a scalable XML-based data dissemination system. Experimental evaluation shows that the size of the routing tables remains small, even with very large populations of consumers.