A wide variety of computer vision applications rely on superpixel or supervoxel algorithms as a preprocessing step. This underlines the overall importance that these approaches have gained in recent years. However, most methods show a lack of temporal consistency or fail in producing temporally stable superpixels. In this paper, we present an approach to generate temporally consistent superpixels for video content. Our method is formulated as a contour-evolving expectation-maximization framework, which utilizes an efficient label propagation scheme to encourage the preservation of superpixel shapes and their relative positioning over time. By explicitly detecting the occlusion of superpixels and the disocclusion of new image regions, our framework is able to terminate and create superpixels whose corresponding image region becomes hidden or newly appears. Additionally, the occluded parts of superpixels are incorporated in the further optimization. This increases the compliance of the superpixel flow with the optical flow present in the scene. Using established benchmark suites, we show the performance of our approach in comparison to state-of-the-art supervoxel and superpixel algorithms for video content.

Many recent superpixel algorithms for video content rely on dense optical flow vectors to propagate segmentation results from one frame to the next. In this paper, we assess the impact of the optical flow quality on the over-segmentation quality. Our evaluation shows that it is indispensable for videos with large object displacement and camera motion. But due to the high computational costs high-quality, dense optical flow is not suitable for real-time applications. Therefore, we propose a fast propagation scheme that is based on sparse feature tracking and mesh-based image warping. In a thorough evaluation, we compare our proposed scheme to the results of other state-of-the-art propagation methods using established benchmarks. The results show that our method speeds up the propagation process by a factor of 100 while producing a comparable segmentation quality.

A major constraint for mobile surveillance systems (e.g. UAV-mounted) is the limited available bandwidth. Thus, ROI-based video coding, where only important image parts are transmitted with high data rate, seems to be a suitable solution for these applications. The challenging part in these systems is to reliably detect the regions of interest (ROI) and to select the corresponding macroblocks to be encoded in high quality. Recently, difference image-based moving object detectors have been used for this purpose. But as these pixel-based approaches fail for homogeneous parts of moving objects, which leads to artifacts in the decoded video stream, this paper proposes the use of a superpixel-based macroblock selection. It is shown that the proposed approach significantly reduces artifacts by raising the moving objects detection rate from 7 % to over 96 % while increasing the data rate by only 20% to 1.5-4Mbit/s (full HD resolution, 30 Hz) still meeting the given constraints. Moreover, other simple approaches (dilation) as well as state-of-the-art GraphCut-based approaches are outperformed.

Super pixel algorithms represent a very useful and increasingly popular preprocessing step for a wide range of computer vision applications, as they offer the potential to boost efficiency and effectiveness. In this regards, this paper presents a highly competitive approach for temporally consistent super pixels for video content. The approach is based on energy-minimizing clustering utilizing a novel hybrid clustering strategy for a multi-dimensional feature space working in a global color subspace and local spatial subspaces. Moreover, a new contour evolution based strategy is introduced to ensure spatial coherency of the generated super pixels. For a thorough evaluation the proposed approach is compared to state of the art super voxel algorithms using established benchmarks and shows a superior performance.

To improve upon the initial disparity estimates stemming from a local correspondence method, a subsequent refinement step is commonly employed. The performance of the stereo matcher used for the initial estimation inherently depends on the underlying image content. For some regions of an image, it may be difficult or even physically impossible to establish accurate point correspondences. This results in disparity estimates of varying accuracy and reliability. In this paper, a confidence map is proposed which combines the consistency and quality of a match. It explicitly models the reliability of each disparity estimate. Such a confidence map represents valuable, additional information that can be leveraged in subsequent steps of the 3D processing chain. In this regard, this paper presents an extension of a cross bilateral filter that leverages this reliability information during a fast-converging refinement step in order to create robust and reliable disparity maps.

This paper describes the work in progress of the European IST-2001-34410 CAMELLIA project, which focuses on a platform-based development of a smart imaging core to be embedded in smart cameras. Therefore the work within the CAMELLIA project comprises the specification and implementation of smart imaging applications including the development of required new algorithms and hardware. Based on an existing video encoding architecture for MPEG-4 Simple Profile, the aim is to design a smart imaging core, which is suitable for automotive and mobile communication applications. Thereby the encoding architecture is to be extended with processing units for low- and mid-level smart imaging functions. To indicate the applicability of this platform-based development, a first approach for a motion estimation based background detection using a hardware motion estimation unit is illustrated in this paper. Furthermore, first results employing the background detection to detect moving objects in video sequences using a functional simulator of a video encoding architecture are presented.