Taking a better look at subjects of interest helps humans to improve confidence in their age estimation. Unlike still images, sequences offer spatio-temporal dynamic information that contains many cues related to age progression. A review of previous work on video-based age estimation indicates that this is an underexplored field of research. This may be caused by a lack of well-defined and publicly accessible video benchmark protocol, as well as the absence of video-oriented training data. To address the former issue, we propose a carefully designed video age estimation benchmark protocol and make it publicly available. To address the latter issue, we design a video-specific age estimation method that leverages pseudo-labeling and semi-supervised learning. Our results show that the proposed method outperforms image-based baselines on both offline and online benchmark protocols, while the online estimation stability is improved by more than 50%.

Unsupervised learning of anomaly detection in high-dimensional data, such as images, is a challenging problem recently subject to intense research. Through careful modelling of the data distribution of normal samples, it is possible to detect deviant samples, so called anomalies. Generative Adversarial Networks (GANs) can model the highly complex, high-dimensional data distribution of normal image samples, and have shown to be a suitable approach to the problem. Previously published GAN-based anomaly detection methods often assume that anomaly-free data is available for training. However, this assumption is not valid in most real-life scenarios, a.k.a. in the wild. In this work, we evaluate the effects of anomaly contaminations in the training data on state-of-the-art GAN-based anomaly detection methods. As expected, detection performance deteriorates. To address this performance drop, we propose to add an additional encoder network already at training time and show that joint generator-encoder training stratifies the latent space, mitigating the problem with contaminated data. We show experimentally that the norm of a query image in this stratified latent space becomes a highly significant cue to discriminate anomalies from normal data. The proposed method achieves state-of-the-art performance on CIFAR-10 as well as on a large, previously untested dataset with cell images.

Current best local descriptors are learned on a large data set of matching and non-matching keypoint pairs. However, data of this kind are not always available, since the detailed keypoint correspondences can be hard to establish. On the other hand, we can often obtain labels for pairs of keypoint bags. For example, keypoint bags extracted from two images of the same object under different views form a matching pair, and keypoint bags extracted from images of different objects form a non-matching pair. On average, matching pairs should contain more corresponding keypoints than non-matching pairs. We describe an end-to-end differentiable architecture that enables the learning of local keypoint descriptors from such weakly labeled data. In addition, we discuss how to improve the method by incorporating the procedure of mining hard negatives. We also show how our approach can be used to learn convolutional features from unlabeled video signals and 3D models.

We address the problem of estimating atmosphere parameters (temperature and water vapor content) from data captured by an airborne thermal hyperspectral imager and propose a method based on linear and nonlinear optimization. The method is used for the estimation of the parameters (temperature and emissivity) of the observed object as well as sensor gain under certain restrictions. The method is analyzed with respect to sensitivity to noise and the number of spectral bands. Simulations with synthetic signatures are performed to validate the analysis, showing that the estimation can be performed with as few as 10-20 spectral bands at moderate noise levels. The proposed method is also extended to exploit additional knowledge, for example, measurements of atmospheric parameters and sensor noise. Additionally, we show how to extend the method in order to improve spectral calibration.

Current best local descriptors are learned on a large dataset of matching and non-matching keypoint pairs. However, data of this kind is not always available since detailed keypoint correspondences can be hard to establish. On the other hand, we can often obtain labels for pairs of keypoint bags. For example, keypoint bags extracted from two images of the same object under different views form a matching pair, and keypoint bags extracted from images of different objects form a non-matching pair. On average, matching pairs should contain more corresponding keypoints than non-matching pairs. We describe an end-to-end differentiable architecture that enables the learning of local keypoint descriptors from such weakly-labeled data.

In recent years, short-term single-object tracking has emerged has a popular research topic, as it constitutes the core of more general tracking systems. Many such tracking methods are based on matching a part of the image with a template that is learnt online and represented by, for example, a correlation filter or a distribution field. In order for such a tracker to be able to not only find the position, but also the scale, of the tracked object in the next frame, some kind of scale estimation step is needed. This step is sometimes separate from the position estimation step, but is nevertheless jointly evaluated in de facto benchmarks. However, for practical as well as scientific reasons, the scale estimation step should be evaluated separately - for example, there might in certain situations be other methods more suitable for the task. In this paper, we describe an evaluation method for scale estimation in template-based short-term single-object tracking, and evaluate two state-of-the-art tracking methods where estimation of scale and position are separable.

Short-term single-object (STSO) tracking in thermal images is a challenging problem relevant in a growing number of applications. In order to evaluate STSO tracking algorithms on visual imagery, there are de facto standard benchmarks. However, we argue that tracking in thermal imagery is different than in visual imagery, and that a separate benchmark is needed. The available thermal infrared datasets are few and the existing ones are not challenging for modern tracking algorithms. Therefore, we hereby propose a thermal infrared benchmark according to the Visual Visual Object Tracking (VOT) protocol for evaluation of STSO tracking methods. The benchmark includes the new LTIR dataset containing 20 thermal image sequences which have been collected from multiple sources and annotated in the format used in the VOT Challenge. In addition, we show that the ranking of different tracking principles differ between the visual and thermal benchmarks, confirming the need for the new benchmark.

Abstract An image mosaic is an assembly of a large number of small images, usually called tiles, taken from a specific dictionary/codebook. When viewed as a whole, the appearance of a single large image emerges, i.e. each tile approximates a small block of pixels. ASCII art is a related (and older) graphic design technique for producing images from printable characters. Although automatic procedures for both of these visualization schemes have been studied in the past, some are computationally heavy and cannot offer real‐time and interactive performance. We propose an algorithm able to reproduce the quality of existing non‐photorealistic rendering techniques, in particular ASCII art and image mosaics, obtaining large performance speed‐ups. The basic idea is to partition the input image into a rectangular grid and use a decision tree to assign a tile from a pre‐determined codebook to each cell. Our implementation can process video streams from webcams in real time and it is suitable for modestly equipped devices. We evaluate our technique by generating the renderings of a variety of images and videos, with good results. The source code of our engine is publicly available.

District heating is a common way of providing heat to buildings in urban areas. The heat is carried by hot water or steam and distributed in a network of pipes from a central power plant. It is of great interest to minimize energy losses due to bad pipe insulation or leakages in such district heating networks. As the pipes generally are placed underground, it may be difficult to establish the presence and location of losses and leakages. Toward this end, this work presents methods for large-scale monitoring and detection of leakages by means of remote sensing using thermal cameras, so-called airborne thermography. The methods rely on the fact that underground losses in district heating systems lead to increased surface temperatures. The main contribution of this work is methods for automatic analysis of aerial thermal images to localize leaking district heating pipes. Results and experiences from large-scale leakage detection in several cities in Sweden and Norway are presented.

We propose a novel method for iterative learning of point correspondences between image sequences. Points moving on surfaces in 3D space are projected into two images. Given a point in either view, the considered problem is to determine the corresponding location in the other view. The geometry and distortions of the projections are unknown, as is the shape of the surface. Given several pairs of point sets but no access to the 3D scene, correspondence mappings can be found by excessive global optimization or by the fundamental matrix if a perspective projective model is assumed. However, an iterative solution on sequences of point-set pairs with general imaging geometry is preferable. We derive such a method that optimizes the mapping based on Neyman's chi-square divergence between the densities representing the uncertainties of the estimated and the actual locations. The densities are represented as channel vectors computed with a basis function approach. The mapping between these vectors is updated with each new pair of images such that fast convergence and high accuracy are achieved. The resulting algorithm runs in real time and is superior to state-of-the-art methods in terms of convergence and accuracy in a number of experiments.

In an effort to overcome the limitations of small target detection in complex urban scene, complementary data sets are combined to provide additional insight about a particular scene. This paper presents a method based on shape/spectral integration (SSI) decision level fusion algorithm to improve the detection of vehicles in semi and deep shadow areas. A four steps process combines high resolution LIDAR and hyperspectral data to classify shadow areas, segment vehicles in LIDAR data, detect spectral anomalies and improves vehicle detection. The SSI decision level fusion algorithm was shown to outperform detection using a single data set and the utility of shape information was shown to be a way to enhance spectral target detection in complex urban scenes.

In this paper, a shadow detection method combining hyperspectral and LIDAR data analysis is presented. First, a rough shadow image is computed through line-of-sight analysis on a Digital Surface Model (DSM), using an estimate of the position of the sun at the time of image acquisition. Then, large shadow and non-shadow areas in that image are detected and used for training a supervised classifier (a Support Vector Machine, SVM) that classifies every pixel in the hyperspectral image as shadow or non- shadow. Finally, small holes are filled through image morphological analysis. The method was tested on data including a 24 band hyperspectral image in the VIS/NIR domain (50 cm spatial resolution) and a DSM of 25 cm resolution. The results were in good accordance with visual interpretation. As the line-of-sight analysis step is only used for training, geometric mismatches (about 2 m) between LIDAR and hyperspectral data did not affect the results significantly, nor did uncertainties regarding the position of the sun.

This paper addresses the three-dimensional (3-D) tracking of pose and animation of the human face in monocular image sequences using active appearance models. The major problem of the classical appearance-based adaptation is the high computational time resulting from the inclusion of a synthesis step in the iterative optimization. Whenever the dimension of the face space is large, a real-time performance cannot be achieved. In this paper, we aim at designing a fast and stable active appearance model search for 3-D face tracking. The main contribution is a search algorithm whose CPU-time is not dependent on the dimension of the face space. Using this algorithm, we show that both the CPU-time and the likelihood of a nonaccurate tracking are reduced. Experiments evaluating the effectiveness of the proposed algorithm are reported, as well as method comparison and tracking synthetic and real image sequences.

In this paper, we consider the potentialities of adapting a 3D deformable face model to video sequences. Two adaptation methods are proposed. The first method computes the adaptation using a locally exhaustive and directed search in the parameter space. The second method decouples the estimation of head and facial feature motion. It computes the 3D head pose by combining: (i) a robust feature-based pose estimator, and (ii) a global featureless criterion. The facial animation parameters are then estimated with a combined exhaustive and directed search. Tracking experiments and performance evaluation demonstrate the feasibility and usefulness of the developed methods. These experiments also show that the proposed methods can outperform the adaptation based on a directed continuous search.

This paper addresses the 3D tracking of pose and animation of the human face in monocular image sequences using deformable 3D models. For each frame, the proposed adaptation is split into two consecutive stages: global and local. In the first stage, the 3D pose of the face is recovered using a RANSAC-based technique involving both the consensus measure and the consistency with a statistical model of a face texture. In the second stage, the local motion associated with some facial features is recovered using the concept of the active appearance model search. Adaptation examples demonstrate the feasibility and robustness of the developed framework.

In model-based, or semantic, coding, parameters describing the nonrigid motion of objects, e.g., the mimics of a face, are of crucial interest. The facial animation parameters (FAPs) specified in MPEG-4 compose a very rich set of such parameters, allowing a wide range of facial motion. However, the FAPs are typically correlated and also constrained in their motion due to the physiology of the human face. We seek here to utilize this spatial correlation to achieve efficient compression. As it does not introduce any interframe delay, the method is suitable for interactive applications, e.g., videophone and interactive video, where low delay is a vital issue.