Light field displays are 3D displays that aim at the continuous reconstruction of a 3D scene by means of a light field reconstruction. Due to a finite number of light rays reproduced in light field displays, their angular and spatial resolutions, also referred to as the display bandwidth, are limited. Display band-width can be evaluated by analytical and experimental methods. In this paper, an efficient experimental method is proposed to estimate the display bandwidth by visualizing a chirp signal on the projection-based light field display and measuring the maximum reproducible display resolution at different distances from the display screen. These measurements are then used to estimate the display's bandwidth. By means of an experiment, it is demonstrated that the proposed method is fast and the bandwidth measurement has an acceptable accuracy.

We address the problem of 3D scene rendering on near-eye integral imaging displays and evaluation of different rendering methods in terms of human perception. We compare three rendering techniques in terms of perceived spatial resolution at different focused depths, simulating the display in virtual environment and representing the eye through a thin-lens camera model.

Shearlet Transform (ST) has been instrumental for the Densely-Sampled Light Field (DSLF) reconstruction, as it sparsifies the underlying Epipolar-Plane Images (EPIs). The sought sparsification is implemented through an iterative regularization, which tends to be slow because of the time spent on domain transformations for dozens of iterations. To overcome this limitation, this letter proposes a novel self-supervised DSLF reconstruction method, CycleST, which employs ST and cycle consistency. Specifically, CycleST is composed of an encoder-decoder network and a residual learning strategy that restore the shearlet coefficients of densely-sampled EPIs using EPI-reconstruction and cycle-consistency losses. CycleST is a self-supervised approach that can be trained solely on Sparsely-Sampled Light Fields (SSLFs) with small disparity ranges (≤8 pixels). Experimental results of DSLF reconstruction on SSLFs with large disparity ranges (16 - 32 pixels) demonstrate the effectiveness and efficiency of the proposed CycleST method. Furthermore, CycleST achieves ~ 9x speedup over ST, at least.

Light field (LF) acquisition devices capture spatial and angular information of a scene. In contrast with traditional cameras, the additional angular information enables novel postprocessing applications, such as 3D scene reconstruction, the ability to refocus at different depth planes, and synthetic aperture. In this paper, we present a novel compression scheme for LF data captured using multiple traditional cameras. The input LF views were divided into two groups: key views and decimated views. The key views were compressed using the multi-view extension of high-efficiency video coding (MV-HEVC) scheme, and decimated views were predicted using the shearlet-transform-based prediction (STBP) scheme. Additionally, the residual information of predicted views was also encoded and sent along with the coded stream of key views. The proposed scheme was evaluated over a benchmark multi-camera based LF datasets, demonstrating that incorporating the residual information into the compression scheme increased the overall peak signal to noise ratio (PSNR) by 2 dB. The proposed compression scheme performed significantly better at low bit rates compared to anchor schemes, which have a better level of compression efficiency in high bit-rate scenarios. The sensitivity of the human vision system towards compression artifacts, specifically at low bit rates, favors the proposed compression scheme over anchor schemes.

Shearlet Transform (ST) is one of the most effective methods for Densely-Sampled Light Field (DSLF) reconstruction from a Sparsely-Sampled Light Field (SSLF). However, ST requires a precise disparity estimation of the SSLF. To this end, in this paper a state-of-the-art optical flow method, i.e. PWC-Net, is employed to estimate bidirectional disparity maps between neighboring views in the SSLF. Moreover, to take full advantage of optical flow and ST for DSLF reconstruction, a novel learning-based method, referred to as Flow-Assisted Shearlet Transform (FAST), is proposed in this paper. Specifically, FAST consists of two deep convolutional neural networks, i.e. disparity refinement network and view synthesis network, which fully leverage the disparity information to synthesize novel views via warping and blending and to improve the novel view synthesis performance of ST. Experimental results demonstrate the superiority of the proposed FAST method over the other state-of-the-art DSLF reconstruction methods on nine challenging real-world SSLF sub-datasets with large disparity ranges (up to 26 pixels).

Shearlet Transform (ST) is one of the most effective algorithms for the Densely-Sampled Light Field (DSLF) reconstruction from a Sparsely-Sampled Light Field (SSLF) with a large disparity range. However, ST requires a precise estimation of the disparity range of the SSLF in order to design a shearlet system with decent scales and to pre-shear the sparsely-sampled Epipolar-Plane Images (EPIs) of the SSLF. To overcome this limitation, a novel coarse-to-fine DSLF reconstruction method, referred to as Mask-Accelerated Shearlet Transform (MAST), is proposed in this paper. Specifically, a state-of-the-art learning-based optical flow method, FlowNet2, is employed to estimate the disparities of a SSLF. The estimated disparities are then utilized to roughly estimate the densely-sampled EPIs for the sparsely-sampled EPIs of the SSLF. Finally, an elaborately-designed soft mask for a coarsely-inpainted EPI is exploited to perform an iterative refinement on this EPI. Experimental results on nine challenging horizontal-parallax real-world SSLF datasets with large disparity ranges (up to 35 pixels) demonstrate the effectiveness and efficiency of the proposed method over the other state-of-the-art approaches.

Light field acquisition technologies capture angular and spatial information of the scene. The spatial and angular information enables various post processing applications, e.g. 3D scene reconstruction, refocusing, synthetic aperture etc at the expense of an increased data size. In this paper, we present a novel prediction tool for compression of light field data acquired with multiple camera system. The captured light field (LF) can be described using two plane parametrization as, L(u, v, s, t), where (u, v) represents each view image plane coordinates and (s, t) represents the coordinates of the capturing plane. In the proposed scheme, the captured LF is uniformly decimated by a factor d in both directions (in s and t coordinates), resulting in a sparse set of views also referred to as key views. The key views are converted into a pseudo video sequence and compressed using high efficiency video coding (HEVC). The shearlet transform based reconstruction approach, presented in [1], is used at the decoder side to predict the decimated views with the help of the key views. Four LF images (Truck, Bunny from Stanford dataset, Set2 and Set9 from High Density Camera Array dataset) are used in the experiments. Input LF views are converted into a pseudo video sequence and compressed with HEVC to serve as anchor. Rate distortion analysis shows the average PSNR gain of 0.98 dB over the anchor scheme. Moreover, in low bit-rates, the compression efficiency of the proposed scheme is higher compared to the anchor and on the other hand the performance of the anchor is better in high bit-rates. Different compression response of the proposed and anchor scheme is a consequence of their utilization of input information. In the high bit-rate scenario, high quality residual information enables the anchor to achieve efficient compression. On the contrary, the shearlet transform relies on key views to predict the decimated views without incorporating residual information. Hence, it has inherit reconstruction error. In the low bit-rate scenario, the bit budget of the proposed compression scheme allows the encoder to achieve high quality for the key views. The HEVC anchor scheme distributes the same bit budget among all the input LF views that results in degradation of the overall visual quality. The sensitivity of human vision system toward compression artifacts in low-bit-rate cases favours the proposed compression scheme over the anchor scheme.

In this article we develop an image based rendering technique based on light field reconstruction from a limited set of perspective views acquired by cameras. Our approach utilizes sparse representation of epipolar-plane images (EPI) in shearlet transform domain. The shearlet transform has been specifically modified to handle the straight lines characteristic for EPI. The devised iterative regularization algorithm based on adaptive thresholding provides high-quality reconstruction results for relatively big disparities between neighboring views. The generated densely sampled light field of a given 3D scene is thus suitable for all applications which require light field reconstruction. The proposed algorithm compares favorably against state of the art depth image based rendering techniques and shows superior performance specifically in reconstructing scenes containing semi-transparent objects.

In this work, we deal with the problem of denoising 3D scene range measurements acquired by Time-of-flight (ToF) range sensors and composed in the form of 2D image-like depth maps. We address the specific case of ToF low-sensing environment (LSE). Such environment is set by low-light sensing conditions, low-power hardware requirements, and low-reflectivity scenes. We demonstrate that data captured by a device in such mode can be effectively post-processed in order to reach the same measurement accuracy as if the device was working in normal operating mode. In order to achieve this, we first present an elaborated analysis of noise properties of ToF data sensed in LSE and verify the derived noise models by empirical measurements. Then, we develop a related novel non-local denoising approach working in complex domain and demonstrate its superiority against the state of the art for data acquired by an off-the-shelf ToF device.

Holographic stereograms (HSs) constitute one of the most widely used types of computer-generated holograms. The scene information required to calculate the HSs can be acquired by conventional digital cameras. It is, however, usually required that the scene should be captured from dense set of view points. Therefore, relieving this requirement is critical in the sense of easing the capture process. In this paper, in the capture stage of holographic stereograms, we employ our previously presented light field reconstruction algorithm [1], where we utilize sparse representation of light fields in the shearlet domain and reconstruct dense light fields from their highly under-sampled versions. The simulation results demonstrate that we can relieve the dense view sampling requirement of HSs, e.g. by as high as 8 × 8 sub-sampling factor, and still keep the perceived image quality of holographic reconstructions at satisfactory levels. This enables, for example, replacing the scanning camera setups with the more convenient multi-camera arrangements.

In this paper we propose a method for reconstructing a densely sampled light field from a given sparse set of perspective views from rectified cameras without an explicit estimation of the scene depth. The desired intermediate views are synthesized by inpainting of epipolar-plane images, utilizing their sparsity in the shearlet domain. For the purpose of shearlet-domain representation, compactly supported shearlets have been constructed using different directional filters for different scales in an attempt to provide better directional selectivity at lower scales. The reconstruction procedure with shearlet-domain sparsity condition is implemented through an iterative thresholding algorithm. The performance of the method is quantified by tests on synthetic and real visual data and compared favorably against depth-image based rendering.

The 2015 edition of the International Symposium on Image and Signal Processing and Analysis (ISPA 2015) is the ninth in the series of biennial research meetings and it follows the successful ISPA 2013 meeting held in Trieste, Italy. ISPA 2015 is held in Zagreb, the capital city of the Republic of Croatia.

Light-field (LF) 3D displays require vast amount of views representing the original scene when using pure light-ray interpolation to convert multi-camera content to display-specific LF representation. Synthetic and real multi-camera setups are both used to feed these displays with image-based data, however the layout, number, frustum, and resolution of these cameras are mostly suboptimal. Storage and transmission of LF data is an issue, especially considering that some of the captured / rendered pixels are left unused while generating the final image. LF displays can have significantly different requirements for camera setups due to differences in Field of View (FOV), angular resolution and spatial resolution. An analysis of typical camera setups and LF display setups, and the typical patterns in pixel usage resulting from the combination of these setups are presented. Based on this analysis, an optimization method for virtual camera setups is proposed. As virtual cameras have wide range of adjustment possibilities, highly optimized setups for specific displays can be achieved.

Effective spatial resolution of projection-based 3D light-field (LF) displays is an important quantity, which is informative about the capabilities of the display to recreate views in space and is important for content creation. We propose a subjective experiment to measure the spatial resolution of LF displays and compare it to our objective measurement technique. The subjective experiment determines the limit of visibility on the screen as perceived by viewers. The test involves subjects determining the direction of patterns that resemble tumbling E eye test charts. These results are checked against the LF display resolution determined by objective means. The objective measurement models the display as a signal-processing channel. It characterizes the display throughput in terms of passband, quantified by spatial resolution measurements in multiple directions. We also explore the effect of viewing angle and motion parallax on the spatial resolution.

The multiple constant multiplication (MCM) implementation of polyphase decomposed finite impulse response (FIR) filters can be formulated as (1) one MCM block with all coefficients, (2) one MCM block for each subfilter, or (3) as a matrix MCM block. The coefficient symmetry of linear phase FIR filters, however, is not exploited in the later two formulations. In this paper, we investigate the MCM implementation of polyphase structures with coefficient symmetry, where each subfilter MCM block is synthesized separately. We show that by using the proposed implementation it is possible to reduce the number of adders compared with currently existing methods without increasing the storage elements. The proposed techniques can also be extended to the matrix MCM block implementation.

This paper proposes a fast design scheme for optimizing a novel family of multi-carrier transmission (MCT) systems that are built up based on critically-sampled cosine-modulated filter banks (CMFBs). This approach extends an earlier-proposed windowing-method (WM)-based technique for optimizing prototype filters for CMFBs such that, in addition to the cut-off frequency of the ideal filter, three terms in a four-term window function are used as unknowns, thereby leading to an optimization problem with only four adjustable parameters. Such an optimization problem is very efficiently solvable also when long prototype filters and many subchannels are required. What makes the resulting MCT systems novel is that the optimization concentrates on minimizing directly a weighted sum of the inter- symbol and inter-channel interferences in these MCT systems, without directly considering the prototype filter. If the weight values in this sum are fixed, then the values of the unknowns can be optimized for the given overlapping factor. After tabulating these values, a closely optimum solution for any number of channels is obtained by simply using the WM together with these tabulated values.

This paper considers how to efficiently implement linear-phase FIR filters for providing a sampling rate conversion by an arbitrary rational factor of M/L, where L(M) is the up-sampling (down-sampling) factor to be implemented before (after) the actual filter. In the proposed implementation, the coefficient symmetry of the linear-phase FIR filter is exploited as well as possible when taking into account the following facts. When increasing (decreasing) the sampling rate by the factor of L(M), only every Lth input sample has a nonzero value (only every Mth output sample has to be evaluated). The proposed implementation is, first, presented by two illustrative examples and, then, guidelines are given on how to efficiently implement a sampling rate converter having an arbitrary rational sampling rate factor M/L

We propose an efficient algorithm for designing the prototype filters of oversampled, near perfect reconstruction (NPR), GDFT modulated, biorthogonal filterbanks with arbitrary delay. Given the analysis prototype, we show that the minimization of the stopband energy of the synthesis prototype, subject to NPR constraints on the frequency response of the distortion transfer function, can be expressed as a convex optimization problem. Our algorithm consists of initialization with the prototype of an orthogonal filterbank and then successive optimization of the synthesis and analysis prototypes. We compare our algorithm with previous methods and give several design examples.

Efficient two-step algorithms are described for optimizing the stopband response of the prototype filter for cosine-modulated and modified DFT filter banks either in the minimax or in the least-mean-square sense subject to the maximum allowable aliasing and amplitude errors. The first step involves finding a good start-up solution using a simple technique. This solution is improved in the second step by using nonlinear optimization. Several examples are included illustrating the flexibility of the proposed approach for making compromises between the required filter lengths and the aliasing and amplitude errors. These examples show that by allowing very small amplitude and aliasing errors, the stopband performance of the resulting filter bank is significantly improved compared to the corresponding perfect-reconstruction filter bank. Alternatively, the filter orders and, consequently, the overall delay can be significantly reduced to achieve practically the same performance.

An efficient iterative method is described for designing orthogonal two-channel perfect-reconstruction FIR filter banks in such a way that the low-pass analysis filter has the given number of fixed zeros at z=-1 and its energy in the given stopband region is minimized. When using the resulting two-channel filter bank for generating discrete-time wavelet banks, the number of vanishing moments is equal to the number of zeros being located at z=-1. The proposed design scheme is fast and the convergence to the optimum solution is independent of the starting-point filter bank. Compared to the two-channel filter bank equivalents designed in the minimax sense as proposed by Rioul and Duhamel (1994), the regularities of the resulting wavelets are increased and the stopband energies of the subfilters are decreased. If there are no constraints on the number of zeros at z=-1, then the resulting banks are useful building blocks in generating frequency-selective multi-channel filter banks and octave filter banks.

An efficient two-step approach is presented for designing two-channel perfect-reconstruction linear-phase FIR filter banks. The first step involves finding a good solution by using an iterative procedure. This iterative procedure is generated by properly modifying the Lagrange-Newton method proposed by Horng and Willson (1992). In the second step, the resulting solution is then used as a good initial solution for further optimization that is carried out by the second algorithm of Dutta and Vidyasagar (1977). Several examples are included illustrating the efficiency of the proposed approach and the resulting two-channel filter banks.