Real-time visual analysis tasks, like tracking and recognition, require swift execution of computationally intensive algorithms. Visual sensor networks could be enabled to perform such tasks by allowing the camera nodes to offload their computational load to nearby processing nodes. In this paper, we address the problem of minimizing the completion time of multiple camera sensors that share the transmission and the processing resources of multiple processing nodes for computation offloading. We show that the problem is NP-hard, and propose a combination of central coordination and distributed optimization with limited signaling among the camera sensors as a solution. We analyze the existence of equilibrium allocations for the distributed algorithms, evaluate the effect of the network topology and of the video characteristics on the algorithms' performance, and assess the benefits of central coordination. Our results demonstrate that with sufficient information available, distributed optimization can provide low completion times, moreover predictable and stable performance can be achieved with additional, sparse central coordination.

We consider the problem of performing distributed visual analysis for a video sequence in a visual sensor network that contains sensor nodes dedicated to processing. Visual analysis requires the detection and extraction of visual features from the images, and thus the time to complete the analysis depends on the number and on the spatial distribution of the features, both of which are unknown before performing the detection. In this paper, we formulate the minimization of the time needed to complete the distributed visual analysis for a video sequence subject to a mean average precision requirement as a stochastic optimization problem. We propose a solution based on two composite predictors that reconstruct randomly missing data, on quantile-based linear approximation of feature distribution and on time series analysis methods. The composite predictors allow us to compute an approximate optimal solution through linear programming. We use two surveillance video traces to evaluate the proposed algorithms, and show that prediction is essential for minimizing the completion time, even if the wireless channel conditions vary and introduce significant randomness. The results show that the last value predictor together with regular quantile-based distribution approximation provide a low complexity solution with very good performance.

Real-time visual analysis tasks, like tracking and recognition, require swift execution of computationally intensive algorithms. Enabling visual sensor networks to perform such tasks can be achieved by augmenting the sensor network with processing nodes and distributing the computational burden among several nodes, in a way that the cameras contend for the processing nodes while trying to minimize their completion times. In this paper, we formulate the problem of minimizing the completion time of all camera sensors as an optimization problem. We propose algorithms for fully distributed optimization, analyze the existence of equilibrium allocations, and evaluate their performance. Simulation results show that distributed optimization can provide good performance despite limited information availability at low computational complexity, but the predictable and stable performance is often not provided by the algorithm that provides lowest average completion time.

Enabling visual sensor networks to perform visual analysis tasks in real-time is challenging due to the computational complexity of detecting and extracting visual features. A promising approach to address this challenge is to distribute the detection and the extraction of local features among the sensor nodes, in which case the time to complete the visual analysis of an image is a function of the number of features found and of the distribution of the features in the image. In this paper we formulate the minimization of the time needed to complete the distributed visual analysis for a video sequence subject to a mean average precision requirement as a stochastic optimization problem. We propose a solution based on two composite predictors that reconstruct randomly missing data, and use a quantile-based linear approximation of the feature distribution and time series analysis methods. The composite predictors allow us to compute an approximate optimal solution through linear programming. We use two surveillance videos to evaluate the proposed algorithms, and show that prediction is essential for controlling the completion time. The results show that the last value predictor together with regular quantile-based distribution approximation provide a low complexity solution with very good performance.

We consider controlling and balancing the processing load in a visual sensor network (VSN) used for detecting local features, such as BRISK. We formulate a prediction problem with random missing data, and propose two regression-based algorithms for data reconstruction. Numerical results illustrate the performance of the proposed algorithms, and show that backward regression combined with the last value predictor can be used for controlling and balancing the processing load in VSNs with good performance.

This demo showcases some of the results obtained by the GreenEyes project, whose main objective is to enable visual analysis on resource-constrained multimedia sensor networks. The demo features a multi-hop visual sensor network operated by BeagleBones Linux computers with IEEE 802.15.4 communication capabilities, and capable of recognizing and tracking objects according to two different visual paradigms. In the traditional compress-then-analyze (CTA) paradigm, JPEG compressed images are transmitted through the network from a camera node to a central controller, where the analysis takes place. In the alternative analyze-then-compress (ATC) paradigm, the camera node extracts and compresses local binary visual features from the acquired images (either locally or in a distributed fashion) and transmits them to the central controller, where they are used to perform object recognition/tracking. We show that, in a bandwidth constrained scenario, the latter paradigm allows to reach better results in terms of application frame rates, still ensuring excellent analysis performance.