The paper describes the implementation and experiences gathered from the Spade project which is a decentralized mechanism for data management that has been used in High Energy and Light Source physics experiments. The background and motivations that shaped the Spade project are covered, followed by an outline of the current design of the deployed application. The operational scenarios that have been developed from the use of Spade in a number of difference contexts are enumerated, after which there is a short discussion on how the implementation of the application has progressed. The paper finishes with some highlight of Spade's performance and a conclusion about what has been learnt from this project.

The advent (and promise) of shared, widely available, high-speed networks provides the potential for new approaches to the collection, organization, storage, and analysis of high-speed and high-volume data streams from high data-rate, on-line instruments. We have worked in this area for several years, have identified and addressed a variety of problems associated with this scenario, and have evolved an architecture, implementations, and a monitoring methodology that have been successful in addressing several different application areas.In this paper we describe a distributed, wide area network-based architecture that deals with data streams that originate from on-line instruments. Such instruments and imaging systems are a staple of modern scientific, health care, and intelligence environments. Our work provides an approach for reliable, distributed real-time analysis, cataloguing, and archiving of the data streams through the integration and distributed management of a high-speed distributed cache, distributed high-performance applications, and tertiary storage systems.