Semi-supervised techniques have been successful in reducing the amount of labels needed to train a neural network. Often these techniques focus on making the most out of the given labels and exploiting the unlabeled data. Instead of considering the labels as a given, we start with focusing on how to select good labels for efficient semi-supervised learning with pseudo-labels. We propose CLaP: Clustering, Label selection and Pseudo-labels. It clusters an unlabeled dataset on an embedding, that was pretrained on large scale dataset (ImageNet1k). We use the cluster centers for querying initial labels that are both representative and diverse. We use samples consistent over multiple clustering runs as pseudo-labels. We propose a mixed loss on both the initial labels and the pseudo-labeled data to train a neural network. With CLaP the samples to be annotated are automatically selected, reducing the load by a human annotator to search for these samples. We demonstrate that CLaP outperforms of state-of-the-art methods in few-shot transfer tasks on full datasets by 20% on 1-shot to 3% on 5-shot. It also outperforms the accuracy of the state-of-the-art in the BSCD-FSL benchmark up to 23%, depending on the dataset and amount of labels.

We present a new family of exchangeable stochastic processes, the Functional Neural Processes (FNPs). FNPs model distributions over functions by learning a graph of dependencies on top of latent representations of the points in the given dataset. In doing so, they define a Bayesian model without explicitly positing a prior distribution over latent global parameters; they instead adopt priors over the relational structure of the given dataset, a task that is much simpler. We show how we can learn such models from data, demonstrate that they are scalable to large datasets through mini-batch optimization and describe how we can make predictions for new points via their posterior predictive distribution. We experimentally evaluate FNPs on the tasks of toy regression and image classification and show that, when compared to baselines that employ global latent parameters, they offer both competitive predictions as well as more robust uncertainty estimates.

In content based video retrieval videos are often indexed with semantic labels ( concepts ) using pre-trained classifiers. These pre-trained classifiers ( concept detectors ), are not perfect, and thus the labels are noisy. Additionally, the amount of pre-trained classifiers is limited. Often automatic methods cannot represent the query adequately in terms of the concepts available. This problem is also apparent in the retrieval of events, such as bike trick or birthday party . Our solution is to obtain user feedback. This user feedback can be provided on two levels: concept level and video level . We introduce the method Adaptive Relevance Feedback ( ARF ) on video level feedback. ARF is based on the classical Rocchio relevance feedback method from Information Retrieval. Furthermore, we explore methods on concept level feedback, such as the re-weighting and Query Point Modification (QPM) methods as well as a method that changes the semantic space the concepts are represented in. Methods on both concept level and video level are evaluated on the international benchmark TRECVID Multimedia Event Detection (MED) and compared to state of the art methods. Results show that relevance feedback on both concept and video level improves performance compared to using no relevance feedback; relevance feedback on video level obtains higher performance compared to relevance feedback on concept level; our proposed ARF method on video level outperforms a state of the art k-NN method, all methods on concept level and even manually selected concepts.

Searching in digital video data for high-level events, such as a parade or a car accident, is challenging when the query is textual and lacks visual example images or videos. Current research in deep neural networks is highly beneficial for the retrieval of high-level events using visual examples, but without examples it is still hard to (1) determine which concepts are useful to pre-train ( Vocabulary challenge ) and (2) which pre-trained concept detectors are relevant for a certain unseen high-level event ( Concept Selection challenge ). In our article, we present our Semantic Event Retrieval System which (1) shows the importance of high-level concepts in a vocabulary for the retrieval of complex and generic high-level events and (2) uses a novel concept selection method ( i-w2v ) based on semantic embeddings. Our experiments on the international TRECVID Multimedia Event Detection benchmark show that a diverse vocabulary including high-level concepts improves performance on the retrieval of high-level events in videos and that our novel method outperforms a knowledge-based concept selection method.

A common approach in content based video information retrieval is to perform automatic shot annotation with semantic labels using pre-trained classifiers. The visual vocabulary of state-of-the-art automatic annotation systems is limited to a few thousand concepts, which creates a semantic gap between the semantic labels and the natural language query. One of the methods to bridge this semantic gap is to expand the original user query using knowledge bases. Both common knowledge bases such as Wikipedia and expert knowledge bases such as a manually created ontology can be used to bridge the semantic gap. Expert knowledge bases have highest performance, but are only available in closed domains. Only in closed domains all necessary information, including structure and disambiguation, can be made available in a knowledge base. Common knowledge bases are often used in open domain, because it covers a lot of general information. In this research, query expansion using common knowledge bases ConceptNet and Wikipedia is compared to an expert description of the topic applied to content-based information retrieval of complex events. We run experiments on the Test Set of TRECVID MED 2014. Results show that 1) Query Expansion can improve performance compared to using no query expansion in the case that the main noun of the query could not be matched to a concept detector; 2) Query expansion using expert knowledge is not necessarily better than query expansion using common knowledge; 3) ConceptNet performs slightly better than Wikipedia; 4) Late fusion can slightly improve performance. To conclude, query expansion has potential in complex event detection.

Many image processing methods, such as techniques for people re‐identification, assume photometric constancy between different images. This study addresses the correction of photometric variations based upon changes in background areas to correct foreground areas. The authors assume a multiple light source model where all light sources can have different colours and will change over time. In training mode, the authors learn per‐location relations between foreground and background colour intensities. In correction mode, the authors apply a double linear correction model based on learned relations. This double linear correction includes a dynamic local illumination correction mapping as well as an inter‐camera mapping. The authors evaluate their illumination correction by computing the similarity between two images based on the earth mover's distance. The authors compare the results to a representative auto‐exposure algorithm found in the recent literature plus a colour correction one based on the inverse‐intensity chromaticity. Especially in complex scenarios the authors’ method outperforms these state‐of‐the‐art algorithms.

Theft of cargo from a truck or attacks against the driver are threats hindering the day to day operations of trucking companies. In this work we consider a system, which is using surveillance cameras mounted on the truck to provide an early warning for such evolving threats. Low-level processing involves tracking people and calculating motion features. Intermediate-level processing provides kinematics and localisation, activity descriptions and threat stage estimates. At the high level, we compare threat detection performed with a statistical trained SVM based classifier against a rule based system. Results are promising, and show that the best system depends on the scenario.

In this paper we present a set of activity recognition and localization algorithms that together assemble a large amount of information about activities on a parking lot. The aim is to detect and recognize events that may pose a threat to truck drivers and trucks. The algorithms perform zone-based activity learning, individual action recognition and group detection. Visual sensor data, from one camera, have been recorded for 23 realistic scenarios of different complexities. The scene is complicated and causes uncertain and false position estimates. We also present a situational assessment ontology which serves the algorithms with relevant knowledge about the observed scene (e.g. information about objects, vulnerabilities and historical data). The algorithms are tested with real tracking data and the evaluations show promising results. The accuracies are 90 % for zone-based activity learning, 71 % for individual action recognition and 66 % for group detection (i.e. merging of people).

In robotic applications the absolute pose is often obtained as the integral of successive relative rigid-body motions. As each relative rigid-body motion is typically the product of statistical inference, the integrated absolute pose will exhibit error build-up and the estimated trajectory will differ from the true trajectory undertaken by the system. Some application areas allow the system to receive additional information about its current absolute pose, for example from loop detection, which is more accurate than the integral of the relative rigid-body motions. The availability of this absolute information is usually less frequent than the information underlying the relative rigid-body motions. This contribution addresses an efficient closed form algorithm which minimally bends a trajectory such that the integrated pose is exactly equal to any particular desired pose. The manner in which the bending is distributed over the trajectory is controllable using weights. The proposed method will be compared against a maximum likelihood solution on simulated trajectories as well as on trajectories estimated from binocular and monocular data. The results indicate that the performance differences between the closed form approach and the maximum likelihood solution are negligible while the closed form approach is significantly more efficient.

Changing image intensities causes problems for many computer vision applications operating in unconstrained environments. We propose generally applicable algorithms to correct for global differences in intensity between images recorded with a static or slowly moving camera, regardless of the cause of intensity variation. The proposed intensity correction is based on intensity-quotient estimation. Various intensity estimation methods are compared. Usability is evaluated with background classification as example application. For this application we introduced the PIPE error measure evaluating performance and robustness to parameter setting. Our approach retains local intensity information, is always operational and can cope with fast changes in intensity. We show that for intensity estimation, robustness to outliers is essential for dynamic scenes. For image sequences with changing intensity, the best performing algorithm (MofQ) improves foreground-background classification results up to a factor two to four on real data.

Multiframe super-resolution (SR) reconstruction of small moving objects against a cluttered background is difficult for two reasons: a small object consists completely of “mixed” boundary pixels and the background contribution changes from frame-to-frame. We present a solution to this problem that greatly improves recognition of small moving objects under the assumption of a simple linear motion model in the real-world. The presented method not only explicitly models the image acquisition system, but also the space-time variant fore- and background contributions to the “mixed” pixels. The latter is due to a changing local background as a result of the apparent motion. The method simultaneously estimates a subpixel precise polygon boundary as well as a high-resolution (HR) intensity description of a small moving object subject to a modified total variation constraint. Experiments on simulated and real-world data show excellent performance of the proposed multiframe SR reconstruction method.

Moving objects are often the most interesting parts in image sequences. When images from a camera are undersampled and the moving object is depicted small on the image plane, processing afterwards may help to improve the visibility as well as automatic recognition of the object. This paper presents an approach which performs Super-Resolution (SR) specifically on small moving objects. The presented approach estimates simultaneously a sub-pixel accurate boundary and an intensity description of the small moving object. This approach has the advantage that it permits explicit modeling of the partial volume effect for pixels which are influenced by both the moving object and the background of the scene. This paper describes the setup of the proposed SR algorithm for small moving objects and shows its superior performance in comparison with a standard SR approach.

In this paper we present a super-resolution scheme specifically designed for faces. First, a face detector is used to find faces in a video frame, after which an optical flow algorithm is applied to track feature points on the faces. Given the set of flow vectors corresponding to a single face, we propose to use the epipolar geometry for rejecting outlying flow vectors. This will improve the registration of the face over multiple frames, and thus lead to an improved super-resolution image. An iterative backprojection method is used for acquiring the super-resolution images

Unlike most Super-Resolution (SR) methods described in literature, which perform only SR reconstruction on the background of an image scene, we propose a framework that performs SR reconstruction simultaneously on the background and on moving objects. After registration of the background, moving objects are detected and to each moving object registration is applied. The fusion and deblurring tasks of the framework are performed by the well-known method of Hardie. To evaluate the performance of the framework the Triangle Orientation Discrimination (TOD) method is used. The TOD method quantitatively measures the SR performance on the image background and on moving objects. From experiments it can be concluded that under proper conditions, the SR performance on moving objects is similar as the SR performance on the background.

The topic of this paper is the integration of expectation maximization (EM) background modeling and template matching using color histograms as templates to improve person tracking for surveillance applications. The tracked objects are humans, which are not rigid bodies. As such shape deformations of the objects must be allowed. For each frame, the decision has to be made which pixels belong to an object, and which do not. The integration of detection and tracking is done using a likelihood-based framework. This way the classification of pixels between background and object can be based on comparing likelihoods rather then separate thresholds. A demonstration of the proposed algorithm is given.

An algorithm is presented that allows one to perform skeletonization of large maps with much lower memory requirements than with the straightforward approach. The maps are divided into overlapping tiles, which are skeletonized separately, using a Euclidean distance transform. The amount of overlap is controlled by the maximum expected width of any map component and the maximum size of what is considered as a small component. Next, the skeleton parts are connected again at the middle of the overlap zones. Some examples are given for efficient memory utilization in tiling an A0 size map into a predefined number of tiles or into tiles of a predefined (square) size. The algorithm is also suited for a parallel implementation of skeletonization.

The problem is to find the correspondence between a digital and a scanned map. The digital map is assumed to be geometrically correct. The hand drawn, scanned mapped is not geometrically correct, but contains information not present in the digital map. The proposed procedure is to analyze the scanned map with the ROCKI system. The output consists of distinct objects, among which houses play an important role. These houses are used to find the correspondence with the digital map. Typical abstractions and drawing errors are circumvented by an inexact matching procedure. It is shown that, based on the expected noise model, the exact match can be found using least squares estimation (LSE).