Autonomous vehicles are equipped with a wide range of sensors and corresponding object detectors tuned to reliably detect vulnerable road users in a variety of conditions. However, the activation scores of these object detectors are easily influenced by contextual factors. To address this challenge, we propose a probabilistic, sensor-agnostic and context-adaptive calibration layer that translates the activation scores of the candidate detections into likelihood ratios that are tuned to that specific context. Our method, seamlessly integrated with the underlying object detector, effectively enhances detection precision by mitigating contextual biases. As a proof of concept, we demonstrate that calibrating the activation scores for four pre-trained state-of-the-art detectors achieves an average precision improvement of up to 4% on the Waymo open dataset for the specific task of pedestrian detection using regular RGB cameras. In challenging scenarios, the average precision can improve up to 9%. Additionally, we showcase that context-calibration emerges as a viable alternative to conventional transfer learning when dealing with limited datasets.

In this paper we propose a computationally efficient neural network for high dynamic range fusion and tone mapping, for application in perception systems of autonomous vehicles. The proposed approach fuses two consecutive, differently exposed images into a single output with good exposure in all regions, in a standard dynamic range. Motion is compensated based on fast optical flow estimation, and subsequently by including an error mask as an input to the network to indicate the remaining artifact-prone regions. This is an efficient way for the network to learn to reduce the ghosting artifacts without increasing computational complexity. Unlike the conventional approach, we train the network on versatile traffic data, and evaluate the performance based on object detection quality metrics, rather than for visual quality. The performance was compared to a similarly complex representative method from literature. We achieved improved performance in challenging light conditions due to the robustness of our method in variable traffic conditions.

We present Quasar, a new programming framework that handles many complex aspects in the design of smart vision systems on embedded platforms, such as parallelization, data flow management, scheduling and load balancing. Quasar, as a high-level programming language, is nearly hardware-agnostic, has a low barrier of entry and is therefore well suited for algorithm design and rapid prototyping. Through several benchmarks and application use cases we demonstrate that programs written in Quasar have a performance that is on a par with (or better than) hand-tuned CUDA and OpenACC code while the development requires much less time and is future-proof.

High Dynamic Range (HDR) displays can show images with higher color contrast levels and peak luminosities than the common Low Dynamic Range (LDR) displays. However, most existing video content is recorded and/or graded in LDR format. To show this LDR content on HDR displays, a dynamic range expansion by using an Inverse Tone Mapped Operator (iTMO) is required. In addition to requiring human intervention for tuning, most of the iTMOs don't consider artistic intentions inherent to the HDR domain. Furthermore, the quality of their results decays with peak brightness above 1000 nits. In this paper, we propose a fully-automatic inverse tone mapping operator based on mid-level mapping. This allows expanding LDR images into HDR with peak brightness over 1000 nits, preserving the artistic intentions inherent to the HDR domain. We assessed our results using full-reference objective quality metrics as HDR- VDP-2.2 and DRIM. Experimental results demonstrate that our proposed method outperforms the current state of the art.

High Dynamic Ranges (HDR) displays can show images with higher color contrast levels and peak luminosities than the commonly used Low Dynamic Range (LDR) displays. Although HDR displays are still expensive, they are reaching the consumer market in the coming years. Unfortunately, most video content is recorded and/or graded in LDR format. Typically, dynamic range expansion by using an Inverse Tone Mapped Operator (iTMO) is required to show LDR content in HDR displays. The most common type of artifact derived from dynamic range expansion is false contouring, which negatively affects the overall image quality. In this paper, we propose a new fast iterative false-contour removal method for inverse tone mapped HDR content. We consider the false-contour removal as a signal reconstruction problem, and we solve it using an iterative Projection Onto Convex Sets (POCS) minimization algorithm. Unlike most other false-contour removal techniques, we define reconstruction constraints taking into account the iTMO used. Experimental results demonstrate the effectiveness of the proposed method to remove false contours while preserving details in the image. In order speed-up the execution time, the proposed method was implemented to run on a GPU. We were able to show that it can be used to remove false contours in real-time from an inverse tone mapped High-definition HDR video sequences at 24 fps.

Recent research in compressed sensing of magnetic resonance imaging (CS-MRI) emphasizes the importance of modeling structured sparsity, either in the acquisition or in the reconstruction stages. Subband coefficients of typical images show certain structural patterns, which can be viewed in terms of fixed groups (like wavelet trees) or statistically (certain configurations are more likely than others). Wavelet tree models have already demonstrated excellent performance in MRI recovery from partial data. However, much less attention has been given in CS-MRI to modeling statistically spatial clustering of subband data, although the potentials of such models have been indicated. In this paper, we propose a practical CS-MRI reconstruction algorithm making use of a Markov random field prior model for spatial clustering of subband coefficients and an efficient optimization approach based on proximal splitting. The results demonstrate an improved reconstruction performance compared with both the standard CS-MRI methods and the recent related methods.

In two-view stereo matching, the disparity of occluded pixels cannot accurately be estimated directly: it needs to be inferred through, e.g., regularisation. When capturing scenes using a plenoptic camera or a camera dolly on a track, more than two input images are available, and - contrary to the two-view case - pixels in the central view will only very rarely be occluded in all of the other views. By explicitly handling occlusions, we can limit the depth estimation of pixel p to only use those cameras that actually observe p. We do this by extending variational stereo matching to multiple views, and by explicitly handling occlusion on a view-by-view basis. Resulting depth maps are illustrated to be sharper and less noisy than typical recent techniques working on light fields.

Wavelet-based demosaicing techniques have the advantage of being computationally relatively fast, while having a reconstruction performance that is similar to state-of-the-art techniques. Because the demosaicing rules are linear, it is fairly simple to integrate denoising into the demosaicing. In this paper, we present a method that performs joint denoising and demosaicing, using a Gaussian Scale Mixture (GSM) prior model, thereby modeling the local edge direction as a hidden variable. The results indicate that this technique offers a better reconstruction performance (in PSNR sense and visually) than sequential demosaicing and denoising. On a recent GPU, our algorithm takes 3.5 s for reconstructing a 12 megapixel RAW digital camera image.

This paper presents a two-stage denoising method for hyper-spectral image (HSI) by combining kernel principal component analysis (KPCA) and total variation (TV). In the first stage, we use KPCA denoising to reduce spectrally uncorre-lated noise. In the second stage, the information content is largely separated from the remaining noise by means of principal component analysis (PCA). The remaining noise is then efficiently removed by fast primal-dual TV denoising in low-energy PCA channels. Experimental results on simulated and real HSIs are very encouraging.

In the field of image denoising, the non-local means (NLMS) filter is a conceptually simple, yet powerful technique. This filter exploits non-local, i.e. spatially repetitive, structure in natural images to estimate noise-free structure. In contrast, a wide variety of image restoration problems have been solved exploiting local smoothness of natural images, e.g. by enforcing sparsity of images when subjected to a multi-resolution transform. In this paper we introduce the prior knowledge of non-local repetitiveness of image structures into a broad multi-resolution image restoration framework. The proposed framework allows the power of the NLMS filter, supplemented by multi-resolution sparsity, to be extended for a wide variety of image restoration problems, such as demosaicing, deconvolution, reconstruction from insufficient measurements,... in a conceptually simple way.

In this paper, we present a first-order primal-dual algorithm for tackling the joint demosaicking and deconvolution problem. The proposed algorithm exploits the sparsity of both discrete gradient (TV) and shearlet coefficients as prior knowledge. In order to deal with this sparsity across the color channels, we first decorrelate the signals in color space before sparsifying them spatially, resulting in a separable transform. We demonstrate that this approach yields better results than employing group sparsity strategies. We propose to update the decorrelation operator during the image reconstruction, this approach will result in a significant improvement in PSNR. By relaxing the sparsity of the chrominance signals, we obtain both better objective and subjective image quality compared to other state-of-the-art demosaicking and deconvolution algorithms. Also, color artifacts due to demosaicking are suppressed very well.

In this paper, we present a new method for High Dynamic Range (HDR) reconstruction based on a set of multiple photographs with different exposure times. While most existing techniques take a deterministic approach by assuming that the acquired low dynamic range (LDR) images are noise-free, we explicitly model the photon arrival process by assuming sensor data corrupted by Poisson noise. Taking the noise characteristics of the sensor data into account leads to a more robust way to estimate the non-parametric camera response function (CRF) compared to existing techniques. To further improve the HDR reconstruction, we adopt the split-Bregman framework and use Total Variation for regularization. Experimental results on real camera images and ground-truth data show the effectiveness of the proposed approach.

Parallel Imaging MRI (pMRI) and Compressive Sensing (CS) are two reconstruction techniques that have recently been applied to increase MRI performance. In this paper we demonstrate that a combined analysis of the pMRI and CS problems leads to a conceptually simple, yet effective technique that outperforms independent approaches to both reconstruction problems. We argue that the proposed technique is also naturally resilient to noise, due to its relation to the MAP image denoising formulation. A modified Basis Pursuit (BP) formulation of the CS-MRI problem allows it to handle the pMRI problem at the same time. We also present an exact solution to this BP problem, using the split Bregman technique, with discrete shearlet transform (DST) regularization. The DST is an excellent choice for natural image applications, due to its optimal sparsity property. Results show that this Compressive Parallel Sensing (COMPASS) reconstruction algorithm outperforms more traditional MRI reconstruction algorithms in both pMRI and CS experiments.