In a weakly-supervised scenario object detectors need to be trained using image-level annotation alone. Since bounding-box-level ground truth is not available, most of the solutions proposed so far are based on an iterative, Multiple Instance Learning framework in which the current classifier is used to select the highest-confidence boxes in each image, which are treated as pseudo-ground truth in the next training iteration. However, the errors of an immature classifier can make the process drift, usually introducing many of false positives in the training dataset. To alleviate this problem, we propose in this paper a training protocol based on the self-paced learning paradigm. The main idea is to iteratively select a subset of images and boxes that are the most reliable, and use them for training. While in the past few years similar strategies have been adopted for SVMs and other classifiers, we are the first showing that a self-paced approach can be used with deep-network-based classifiers in an end-to-end training pipeline. The method we propose is built on the fully-supervised Fast-RCNN architecture and can be applied to similar architectures which represent the input image as a bag of boxes. We show state-of-the-art results on Pascal VOC 2007, Pascal VOC 2010 and ILSVRC 2013. On ILSVRC 2013 our results based on a low-capacity AlexNet network outperform even those weakly-supervised approaches which are based on much higher-capacity networks.

We address the problem of automatic extraction of foreground objects from videos. The goal is to provide a method for unsupervised collection of samples which can be further used for object detection training without any human intervention. We use the well known Selective Search approach to produce an initial still-image based segmentation of the video frames. This initial set of proposals is pruned and temporally extended using optical flow and transductive learning. Specifically, we propose to use Dense Trajectories in order to robustly match and track candidate boxes over different frames. The obtained box tracks are used to collect samples for unsupervised training of track-specific detectors. Finally, the detectors are run on the videos to extract the final tubes. The combination of appearance-based static "objectness" (Selective Search), motion information (Dense Trajectories) and transductive learning (detectors are forced to "overfit" on the unsupervised data used for training) makes the proposed approach extremely robust. We outperform state-of-the-art systems by a large margin on common benchmarks used for tube proposal evaluation.

Color plays an essential role in everyday life and is one of the most important visual cues in human perception. In abstract art, color is one of the essential means to convey the artist's intention and to affect the viewer emotionally. However, colors are rarely experienced in isolation, rather, they are usually presented together with other colors. In fact, the expressive properties of two-color combinations have been extensively studied by artists. It is intriguing to try to understand how color combinations in abstract paintings might affect the viewer emotionally, and to investigate if a computer algorithm can learn this mechanism.

The paper addresses the problem of learning features that can account for temporal dynamics present in videos. Although deep convolutional learning methods revolutionized several areas of multimedia and computer vision, there have been relatively few proposals dealing with ways in which these methods can be enabled to make use of motion information, critical to the extraction of useful information from video. We propose a temporal dropout of changes approach for this, which allows us to consider temporal information over a series of frames without increasing the number of training parameters of the network.

In this paper we investigate the design of DASH stream-switching solution suitable for mobile cellular video streaming consumers. Our goal is to design a solution that reduces the amount of downloaded content while adhering to user preferences regarding received video quality. We firstly present a graphical (trellis-like) representation of a stream-switching process, and then investigate different approaches to search for an optimal path through a “stream-switching graph” using suitably defined utility function. We target our search towards the path that will: i) respect underlying wireless-link rate limitations, ii) maintain segment selection that is efficient in terms of the quality vs the amount of the downloaded data, and iii) reduce disturbing quality variations. The simulation results demonstrate the performance of the proposed QoE-aware Rate-Conservative (QaRC) DASH as compared to the recently proposed baseline stream-switching solution.

Design of algorithms that are able to estimate video quality as perceived by human observers is of interest for a number of applications. Depending on the video content, the artifacts introduced by the coding process can be more or less pronounced and diversely affect the quality of videos, as estimated by humans. While it is well understood that motion affects both human attention and coding quality, this relationship has only recently started gaining attention among the research community, when video quality assessment (VQA) is concerned. In this paper, the effect of calculating several objective measure features, related to video coding artifacts, separately for salient motion and other regions of the frames of the sequence is examined. In addition, we propose a new scheme for quality assessment of coded video streams, which takes into account salient motion. Standardized procedure has been used to calculate the Mean Opinion Score (MOS), based on experiments conducted with a group of non-expert observers viewing standard definition (SD) sequences. MOS measurements were taken for nine different SD sequences, coded using MPEG-2 at five different bit-rates. Eighteen different published approaches related to measuring the amount of coding artifacts objectively on a single-frame basis were implemented. Additional features describing the intensity of salient motion in the frames, as well as the intensity of coding artifacts in the salient motion regions were proposed. Automatic feature selection was performed to determine the subset of features most correlated to video quality. The results show that salient-motion-related features enhance prediction and indicate that the presence of blocking effect artifacts and blurring in the salient regions and variance and intensity of temporal changes in non-salient regions influence the perceived video quality.

Although used extensively for object recognition, texture has for the most part been ignored as a feature used for background modelling and object segmentation. The complexity of working with texture descriptors for segmentation in videos is two-fold: the descriptive features cannot be calculated in real time and features extracted based on arbitrarily chosen regions or blocks in the frame are not stable enough to allow for building models sufficiently accurate, yet simple enough to be used for real-time segmentation. The paper proposes an approach that can be used to detect regions of texture, stable enough to be modelled using probabilistic models commonly used for foreground segmentation. Based on the evaluated stable texture regions, a discriminative texture descriptor is proposed that can be evaluated in real time. When video is captured from a stationary camera, the background is expected to be stationary to a degree and an adaptive model can be built to serve as basis for segmentation. For reasons of efficiency most adaptive models are learnt on a per pixel basis [4]. Joint domain-space modelling [3], as well as modelling of the sequence at two different scales, has been suggested as a way to enhance the segmentation and escape the limitations of single-pixel-based models[1]. The improved approaches still fail to consider the texture properties of the objects in the background and suffer from the noise introduced by neighboring objects being modelled as one. Texture of the background can be used to build models in a more informed way. Large stable regions in the texture of the background allow for larger scale modelling and more precise extraction of texture related features. Conversely, unstable regions in terms of texture correspond to transitions between different objects in the background, and the features should be extracted for each of the objects separately. An approach to background modelling and foreground segmentation derived form these principles is proposed in the paper. In each frame of the sequence stable texture regions are determined, texture descriptor (feature) values extracted using the stable regions information and used to model the background and segment the foreground using standard probabilistic approaches. The process is illustrated in Fig. 1, where the brighter patches in the stability map correspond to more stable regions, in terms of texture. Once the size of the stable-texture region at each location is determined, arbitrary descriptive features can be extracted from the frames of the sequence. In addition, one can choose any of a number of probabilistic methods [4][2] to learn the statistics of the features. The size of the stable texture regions can be determined online, using integral images [6]. Observe that:

The paper presents an efficient and reliable approach to automatic people segmentation, tracking and counting, designed for a system with an overhead mounted (zenithal) camera. Upon the initial block-wise background subtraction, k-means clustering is used to enable the segmentation of single persons in the scene. The number of people in the scene is estimated as the maximal number of clusters with acceptable inter-cluster separation. Tracking of segmented people is addressed as a problem of dynamic cluster assignment between two consecutive frames and it is solved in a greedy fashion. Systems for people counting are applied to people surveillance and management and lately within the ambient intelligence solutions. Experimental results suggest that the proposed method is able to achieve very good results in terms of counting accuracy and execution speed.

In real-world surveillance systems, where variation of light and camera parameters can sometimes severely impair the normal operation of background subtraction algorithms, better results are obtained with differencing schemes. We have earlier demonstrated that differencing of detail images produced by wavelet transformation can lead to more stable detection results. In this paper, we considerably extend that framework, by introducing the modified z-scores calculated from wavelet coefficient differences. Foreground pixels are detected as outliers in normal distribution by modified z-score test. The threshold value used in the outlier test is optimized by maximizing the precision and recall measures on several training frames. Finally, the elimination of ghosts from motion detection is done by double modified z-score testing, that is similar in idea to double frame differencing. The resulting motion detection method shows considerable resilience to changes in illumination and camera parameters and also produces a lower amount of detection errors than other motion detection methods.

A novel encryption model for digital videos is presented. The model relies on the encryption-compression duality of certain types of permutations acting on video frames. In essence, the proposed encryption process preserves the spatial correlation and, as such, can be applied prior to the compression stage of a spatial-only video encoder. Several algorithmic modes of the proposed model targeted for different application requirements are presented and analyzed in terms of security and performance. Experimental results are generated for a number of standard benchmark sequences showing that the proposed method, in addition to providing confidentiality, preserves or improves the compression ratio. 1

This paper presents a novel background modeling and subtraction approach for video object segmentation. A neural network (NN) architecture is proposed to form an unsupervised Bayesian classifier for this application domain. The constructed classifier efficiently handles the segmentation in natural-scene sequences with complex background motion and changes in illumination. The weights of the proposed NN serve as a model of the background and are temporally updated to reflect the observed statistics of background. The segmentation performance of the proposed NN is qualitatively and quantitatively examined and compared to two extant probabilistic object segmentation algorithms, based on a previously published test pool containing diverse surveillance-related sequences. The proposed algorithm is parallelized on a subpixel level and designed to enable efficient hardware implementation.

Low bitrates enabled by the H.264/AVC standard come at the cost of significant decoding complexity. The problem is exacerbated when multi-view video coding (MVC), presently being standardized by the MPEG committee, is considered. Increasing the number of reference views will improve the prediction but also increase the encoding and decoding complexity. In this paper we present an alternative to transform coding and arithmetic coding used to code motion compensation residuals in H.264/AVC. Proposed approach is sorting permutation based, resulting in significantly decreased decoding complexity. We evaluate the efficiency of the coding and present initial results obtained for some sequences in published literature