Light field cameras and multi-camera arrays have emerged as promising solutions for accurately estimating depth by passively capturing light information. This is possible because the 3D information of a scene is embedded in the 4-D light field geometry. Commonly, depth estimation methods extract this information relying on gradient information, heuristic-based optimisation models, or learning-based approaches. This paper focuses mainly on explicitly understanding and exploiting 4-D geometrical cues for light field depth estimation. Thus, a novel method is proposed, based on a non-learning-based optimisation approach for depth estimation that explicitly considers surface normal accuracy and occlusion regions by utilising a fully explainable 4-D geometric model of the light field. The 4-D model performs depth/disparity estimation by determining the orientations and analysing the intersections of key 2D planes in 4-D space, which are the images of 3D-space points in the 4-D light field. Experimental results show that the proposed method outperforms both learning-based and non-learning-based state-of-the-art methods in terms of surface normal angle accuracy, achieving a Median Angle Error on planar surfaces, on average, 26.3% lower than the state-of-the-art, and still being competitive with state-of-the-art methods in terms of MSE $\boldsymbol {\times } 100$ and Badpix 0.07.

Wavelet transforms have been a relevant topic in signal processing for many years. One of the most common strategies when designing wavelet transforms is the use of lifting schemes, known for their perfect reconstruction properties and flexible design. This paper introduces a novel 2D non-separable lifting design methodology based on deep learning architectures. The proposed method is assessed within the context of end-to-end lossless image compression. The attained results highlight a significant 99% reduction of the number of learnable coefficients, with a direct impact on the computational complexity, while still achieving higher compression performance when compared to other end-to-end state-of-the-art wavelet-based frameworks. The code for the models used to obtain the reported results are available at https://github.com/joaoparracho/2D-NSWT-LCLS.

Common representations of light fields use four-dimensional data structures, where a given pixel is closely related not only to its spatial neighbours within the same view, but also to its angular neighbours, co-located in adjacent views. Such structure presents increased redundancy between pixels, when compared with regular single-view images. Then, these redundancies are exploited to obtain compressed representations of the light field, using prediction algorithms specifically tailored to estimate pixel values based on both spatial and angular references. This paper proposes new encoding schemes which take advantage of the four-dimensional light field data structures to improve the coding performance of Minimum Rate Predictors. The proposed methods expand previous research on lossless coding beyond the current state-of-the-art. The experimental results, obtained using both traditional datasets and others more challenging, show bit-rate savings no smaller than 10%, when compared with existing methods for lossless light field compression.

This paper describes a robust depth estimation algorithm for multi-focus plenoptic images. The main feature of the proposed method consists of a hybrid template matching scheme built-upon intensity and local phase information, which adapts to the blurriness of neighbouring lenslet microimages. By reducing the impact of defocusblur on the template matching accuracy, the proposed method efficiently handles the varying triangulation baseline over the depth-of-field, thus discarding the need for scene-related information such as the expected range of disparities. Experimental results demonstrate the robustness of the proposed method over the most used commercially available depth estimation algorithm, achieving a reduction of 73% on the depth estimation error.

Efficient light field encoders take advantage of the inherent 4D data structures to achieve high compression performance. This is accomplished by exploiting the redundancy of co-located pixels in different sub-aperture images (SAIs) through prediction and/or transform schemes to find a m ore compact representation of the signal. However, in image regions with higher disparity between SAIs, such scheme's performance tends to decrease, thus reducing the compression efficiency. This paper introduces a reversible pre-processing algorithm for disparity compensation that operates on the SAI domain of light field data. The proposed method contributes to improve the transform efficiency of the encoder, since the disparity-compensated data presents higher correlation between co-located image blocks. The experimental results show significant improvements in the compression performance of 4D light fields, achieving Bjontegaard delta rate gains of about 44% on average for MuLE codec using the 4D discrete cosine transform, when encoding High Density Camera Arrays (HDCA) light field images.

This letter proposes a method to obtain integer reversible discrete cosine transforms for generic transform-based coding schemes. The novelty of the proposed method, which is based on decomposition of the DCT-II matrix into two triangular and one diagonal matrices, is twofold: (i) the new matrices can be of arbitrary size, i.e., any square N × N dimension, thus suitable for applications where non power-of-2 dimensions are required; (ii) they can be designed with adjustable precision in a trade-off with the number of representation bits. Furthermore, improvements are also proposed over the base scheme to avoid numerical issues when working with large matrices and to obtain more reliable approximations. The performance evaluation demonstrate the effectiveness of the proposed transforms to approximate the coding gain capabilities of the original DCT-II.

This paper presents a special matrix factorization based on sparse representation that detects anomalies in video sequences generated with moving cameras. Such representation is made by associating the frames of the target video, that is a sequence to be tested for the presence of anomalies, with the frames of an anomaly-free reference video, which is a previously validated sequence. This factorization is done by a sparse coefficient matrix, and any target-video anomaly is encapsulated into a residue term. In order to cope with camera trepidations, domaintransformations are incorporated into the sparse representation process. Approximations of the transformed-domain optimization problem are introduced to turn it into a feasible iterative process. Results obtained from a comprehensive video database acquired with moving cameras on a visually cluttered environment indicate that the proposed algorithm provides a better geometric registration between reference and target videos, greatly improving the overall performance of the anomaly-detection system.

This paper deals with the challenging problem of visual anomaly detection in a cluttered environment using videos acquired with a moving camera. The anomalies considered are abandoned objects. A new method is proposed for comparing two videos (an anomaly-free reference video and a target one possibly with anomalies) by using convolutional neural networks as feature extractors for a subsequent anomaly-detection stage using a classifier. Two classifier strategies are considered, namely a fully-connected neural network and a random forest algorithm. Results for a comprehensive abandoned object database acquired with a moving camera in a cluttered environment indicate that the proposed architecture can match even the state-of-the-art algorithms in terms of object-detection performance, with a reduction in processing time of 80%.

The detection of abandoned objects in videos from moving cameras is of great importance to automatic surveillance systems that monitor large and visually complex areas. This paper proposes a new method based on sparse decompositions to identify video anomalies associated with abandoned objects. The proposed scheme inherently incorporates synchronization between the reference (anomaly-free) and target (under analysis) sequences thus reducing the implementation complexity of the overall surveillance system. Results indicate that the proposed video-processing scheme can lead to 95% complexity reduction while maintaining excellent detection capability of foreground objects.

This work presents a video-based sequence synchronization algorithm to be used in real-time video surveillance applications. The signals are aligned based on an online dynamic time warping approach that uses only video content information. The algorithm was tested in the alignment of reference and target videos acquired in a cluttered industrial environment with a moving camera. During each recording, the camera follows approximately the same trajectory but its speed along the trajectory may differ among recordings, thus generating time warping between the videos. The results shows that, when considering both alignment error and computational complexity, the use of the mean squared error as similarity metric leads to the best performance among metrics such as the SSIM, absolute error or even sophisticated matching techniques such as histograms of oriented gradients. The proposed method was also tested in a real-time application. Results show that the instantaneous alignment computation of a target frame can be performed successfully, at the cost of a three-fold increase in the error when compared to the case when latency is allowed.

This work presents a novel approach to detect automatically the position of dark fringes in birefringence images. These positions are important for rheology (study of the flow of matter) applications, as they allow indirect measurements of physical properties of molten polymers without having to interact with the material. Our approach uses mathematical morphology techniques to find the patterns that characterize the fringes and detect the center position of each one of them. The results are compared with manual measurements, revealing the superior precision of our method. They have also been applied to the calculation of the stress field of some polymers with good results.

This work presents a framework to be used in the detection of abandoned objects and other video events in a cluttered environment with a moving camera. In the proposed method a target video, that may have features we would like to detect, is compared with a pre-acquired reference video, which is assumed to have no objects nor video events of interest. The comparison is carried out by way of the achieved optimized operators, generated from the reference video, that produce Gaussian outputs when applied to it. Any anomaly of interest in the target video leads to a non-Gaussian output. The method dispenses with the target and reference videos being either synchronized or precisely registered, being robust to rotations and translations between the frames. Experiments show its good performance in the proposed environment.