Digital technologies reshape the way we interact with our environment, including with artworks. Advanced computational imaging solutions allow having extremely high-resolution digital reproductions of artworks outside museums, presumably increasing artwork engagement. We tested whether exploring such reproductions via an interactive interface heightened aesthetic appreciation and enhanced recognition. With this interface, observers can move, turn, zoom, and relight the digital reproductions (http://artmyn.com/; https://osf.io/3srfw/). In Study 1, 82 participants explored paintings in the laboratory. In Study 2, 63 participants explored precious cultural artefacts in the museum. In both studies, participants’ aesthetic appreciation (interest, pleasure, intensity, subjective learning) was higher towards artworks they had explored interactively as compared to non-interactively or as physical objects, highlighting the advantage of the tested technology. However, we found no evidence that interactive exploration improved artwork recognition. More studies are needed to learn when and why digital interaction is beneficial above and beyond subjective aesthetic evaluations.

We present a new technique for estimating the specular peak of the bidirectional reflectance distribution function (BRDF) based on finite rate of innovation (FRI) sampling. The specular component of the BRDF varies rapidly, so it is challenging to acquire it by point-wise sampling. Yet, the knowledge of its precise location is key to render realistically complex materials. We show how to adapt the FRI framework to accurately determine the location of a single pulse when the sampling kernel is unknown. We use this result to determine the position of the specularity, and then estimate its shape by non-linear optimization. We demonstrate the feasibility of our approach in simulations and via a practical experiment using a custom-built BRDF acquisition device.

Continuous-domain visual signals are usually captured as discrete (digital) images. This operation is not invertible in general, in the sense that the continuous-domain signal cannot be exactly reconstructed based on the discrete image, unless it satisfies certain constraints (e.g., bandlimitedness). In this paper, we study the problem of recovering shape images with smooth boundaries from a set of samples. Thus, the reconstructed image is constrained to regenerate the same samples (consistency), as well as forming a shape (bilevel) image. We initially formulate the reconstruction technique by minimizing the shape perimeter over the set of consistent binary shapes. Next, we relax the non-convex shape constraint to transform the problem into minimizing the total variation over consistent non-negative-valued images. We also introduce a requirement (called reducibility) that guarantees equivalence between the two problems. We illustrate that the reducibility property effectively sets a requirement on the minimum sampling density. We also evaluate the performance of the relaxed alternative in various numerical experiments.

Stained glass windows are designed to reveal their powerful artistry under diverse and time-varying lighting conditions; virtual relighting of stained glass, therefore, represents an exceptional tool for the appreciation of this age old art form. However, as opposed to most other artifacts, stained glass windows are extremely difficult if not impossible to analyze using controlled illumination because of their size and position. In this paper, we present novel methods built upon image based priors to perform virtual relighting of stained glass artwork by acquiring the actual light transport properties of a given artifact. In a preprocessing step, we build a material-dependent dictionary for light transport by studying the scattering properties of glass samples in a laboratory setup. We can now use the dictionary to recover a light transport matrix in two ways: under controlled illuminations the dictionary constitutes a sparsifying basis for a compressive sensing acquisition, while in the case of uncontrolled illuminations the dictionary is used to perform sparse regularization. The proposed basis preserves volume impurities and we show that the retrieved light transport matrix is heterogeneous, as in the case of real world objects. We present the rendering results of several stained glass artifacts, including the Rose Window of the Cathedral of Lausanne, digitized using the presented methods.

The spatial frequency response (SFR) is one of the most important and unbiased image quality measures of a digital camera. It evaluates to which extent a lens/sensor combination can resolve scene details. In this paper, we propose a simple and practical method to measure the SFR of microlens-based light field cameras. The particularity of such cameras resides in their ability to capture both spatial and angular information of the incoming light field thanks to an array of microlenses located in front of the sensor. Existing methods for measuring the SFR of conventional cameras are thus no longer applicable as the interaction between the main lens and the micro-lenses results in different resolving powers over the image plane that depend on the scene depths. By using a 3-dimensional target made of vertical lines printed on an inclined planar surface, we are able to measure the SFR across multiple depths in a single exposure. Our method allows SFR measurements from the raw light field itself as captured by the camera, and is thus independent of subsequent post-processing algorithms such as image reconstruction, digital refocusing or depth estimation. Our experimental results are consistent with theoretical bounds and reproducible.

In this letter, we present a possible extension of the theory of sampling signals with finite rate of innovation (FRI) to the case of multichannel acquisition systems. The essential issue of a multichannel system is that each channel introduces different unknown delays and gains that need to be estimated for the calibration of the channels. We pose both the synchronization stage and the signal reconstruction stage as a parametric estimation problem and demonstrate that a simultaneous exact synchronization of the channels and reconstruction of the FRI signal is possible. We also consider the case of noisy measurements and evaluate the Cramér–Rao bounds (CRB) of the proposed system. Numerical results as well as the CRB show clearly that multichannel systems are more resilient to noise than the single-channel ones.

The accurate registration of multiview images is of central importance in many advanced image processing applications. Image super-resolution, for example, is a typical application where the quality of the super-resolved image is degrading as registration errors increase. Popular registration methods are often based on features extracted from the acquired images. The accuracy of the registration is in this case directly related to the number of extracted features and to the precision at which the features are located: images are best registered when many features are found with a good precision. However, in low-resolution images, only a few features can be extracted and often with a poor precision. By taking a sampling perspective, we propose in this paper new methods for extracting features in low-resolution images in order to develop efficient registration techniques. We consider, in particular, the sampling theory of signals with finite rate of innovation and show that some features of interest for registration can be retrieved perfectly in this framework, thus allowing an exact registration. We also demonstrate through simulations that the sampling model which enables the use of finite rate of innovation principles is well suited for modeling the acquisition of images by a camera. Simulations of image registration and image super-resolution of artificially sampled images are first presented, analyzed and compared to traditional techniques. We finally present favorable experimental results of super-resolution of real images acquired by a digital camera available on the market.

Super-resolution algorithms combine multiple low resolution images into a single high resolution image. They have received a lot of attention recently in various application domains such as HDTV, satellite imaging, and video surveillance. These techniques take advantage of the aliasing present in the input images to reconstruct high frequency information of the resulting image. One of the major challenges in such algorithms is a good alignment of the input images: subpixel precision is required to enable accurate reconstruction. In this paper, we give an overview of some subspace techniques that address this problem. We first formulate super-resolution in a multichannel sampling framework with unknown offsets. Then, we present three registration methods: one approach using ideas from variable projections, one using a Fourier description of the aliased signals, and one using a spline description of the sampling kernel. The performance of the algorithms is evaluated in numerical simulations.

The problem of image super-resolution from a set of low resolution multiview images has recently received much attention and can be decomposed, at least conceptually, into two consecutive steps as: registration and restoration. The ability to accurately register the input images is key to the success and the quality of image super-resolution algorithms. Using recent results from the sampling theory for signals with finite rate of innovation (FRI), we propose in this paper a new technique for subpixel extraction from low resolution images of local features like step edges and corners for image registration. By exploiting the knowledge of the sampling kernel, we are able to locate exactly the step edges on synthetic images. We also present results of full frame super-resolution of real low resolution images using our registration technique. We obtain super-resolved images with a much improved visual quality compared to using a standard local feature detection approach like a subpixel Harris corner detector.

Recently, new sampling schemes were presented for signals with finite rate of innovation (FRI) using sampling kernels reproducing polynomials or exponentials. In this paper, we extend those sampling schemes to a distributed acquisition architecture in which numerous and randomly located sensors are pointing to the same area of interest. We emphasize the importance played by moments and show how to acquire efficiently FRI signals with a set of sensors. More importantly, we also show that those sampling schemes can be used for accurate registration of affine transformed and low-resolution images. Based on this, a new super-resolution algorithm was developed and showed good preliminary results.