Complex media fusion operations can be costly in terms of the time they need to process input objects. If data arrive faster to fusion nodes than the speed with which they can consume the inputs, this will result in some input objects not being processed. In this paper, we develop load shedding mechanisms which take into consideration both data quality and expensive nature of media fusion operators. In particular, we present quality assessment models for objects and multistream fusion operators and highlight that such quality assessments may impose partial orders on objects. We highlight that the most effective load control approach for fusion operators involves shedding of (not the individual input objects but) combinations of objects. Yet, identifying suitable combinations of objects in real time will not be possible if efficient combination selection algorithms do not exist. We develop efficient combination selection schemes for scenarios with different quality assessment and target characteristics. We first develop efficient combination-based load shedding when the fusion operator has unambiguously monotone semantics. We then extend this to the more general ambiguously monotone case and present experimental results that show the performance gains using quality-aware combination-based load shedding strategies under the various fusion scenarios.

In situ wireless sensor networks, not only have to route sensed data from sources to destinations, but also have to filter and fuse observations to eliminate potentially irrelevant data. If data arrive faster to such fusion nodes than the speed with which they can consume the inputs, this will result in an overflow of input buffers. In this paper, we develop load shedding mechanisms which take into consideration both data quality and expensive nature of fusion operators. In particular, we present quality assessment models for objects and fusion operators and we highlight that such quality assessments may impose partial orders on objects.

Media-rich ubiquitous distributed media processing workflow systems continuously sense users' needs, status, and the context, filter and fuse a multitude of real-time media data, and react by adapting the environment to the user. One challenge facing these systems is that they need to process real-time data arriving continuously from the sensors and data rates and qualities may vary dramatically. Thus, reducing the amount of unqualified data objects that need to be processed within the underlying media processing workflows can enhance the performance significantly. In this paper, we first focus on the prediction of the output qualities at the actuator, especially in the presence of fusion operators in the workflow. We then present quality-aware early object elimination schemes to enable informed resource savings in continuous real-time media processing workflow middleware.

ARIA, architecture for interactive arts, is a middleware to process, filter, and fuse sensory inputs and actuate responses in real-time. An ARIA media processing workflow describes how the data sensed through media will be processed and what audio-visual responses will be actuated. Each object streamed between ARIA processing components is subject to transformations, as described by a media workflow graph. The media capture and processing components, such as media filters and fusion operators, are programmable and adaptable; i.e, the delay, size, frequency, and quality/precision characteristics of individual operators can be controlled via a number of parameters. In this paper, we present the underlying model which captures the dynamic nature of the ARIA media processing workflows. This model enables design time verification, optimization, and runtime adaptation

Recently, we introduced a novel ARchitecture for Interactive Arts (ARIA) middleware that processes, filters, and fuses sensory inputs and actuates responses in real-time while providing various Quality of Service (QoS) guarantees. The objective of ARIA is to incorporate realtime, sensed, and archived media and audience responses into live performances, on demand. An ARIA media workflow graph describes how the data sensed through media capture devices will be processed and what audio-visual responses will be actuated. Thus, each data object streamed between ARIA processing components is subject to transformations, as described by a media workflow graph. The media capture and processing components, such as media filters and fusion operators, are programmable and adaptable; i.e, the delay, size, frequency, and quality/precision characteristics of individual operators can be controlled via a number of parameters. In [1, 4, 5], we developed static and dynamic optimization algorithms which maximize the quality of the actuated responses, minimize the corresponding delay and the resource usage. In this demonstration, we present the ARIA GUI and the underlying kernel. More specifically, we describe how to design a media processing workflow, with adaptive operators, using the ARIA GUI and how to use the various optimization and adaptation alternatives provided by the ARIA kernel to execute media processing workflows.

We are developing an adaptive and programmable media-flow ARchitecture for Interactive Arts (ARIA) to enable real-time control of audio, video, and lighting on an intelligent stage. The intelligent stage is being equipped with a matrix of floor sensors for object localization, microphone arrays for sound localization, beam forming and motion capture system. ARIA system provides an interface for specifying intended mappings of the sensory inputs to audio-visual responses. Based on the specifications, the sensory inputs are streamed, filtered and fused, actuate a controllable projection system, sound surround and lighting system. The actuated responses take place in real-time and satisfy QoS requirements in live performance. In this paper, we present the ARIA quality-adaptive architecture. We model the basic information unit as a data object with a meta-data header and object payload streamed between nodes in the system and use a directed acyclic network to model media stream processing. We define performance metrics for the output precision, resource consumption, and end-to-end delay. The filters and fusion operators are being implemented by quality aware signal processing algorithms. The proper node behavior is chosen at runtime to achieve the QoS requirements and adapt to input object properties. For this purpose, ARIA utilizes a two-phase approach: static pre-optimization and dynamic run-time adaptation.