Estimation of depth in two-dimensional images is among the challenging topics in Computer Vision. This is a well-studied but also an ill-posed problem, which has long been the focus of intense research. This paper is an in-depth review of the topic, presenting two aspects, one that considers the mechanisms of human depth perception, and another that includes the various Deep Learning approaches. The methods are presented in a compact and structured way that outlines the topic and categorizes the approaches according to the line of research followed in the recent decade. Although there has been significant advancement in the topic, it was without any connection with human depth perception and the potential benefits from this sector.

Peripheral oxygen saturation (SpO2) is one important vital sign to be monitored in individuals, whose health is fragile, such as the elderly. Contactless SpO2monitoring using RGB cameras has been already developed with satisfactory results. This work explores the case of achieving an acceptable level of performance, when the lightning conditions are not optimal, particularly during night time, by processing solely infrared low-cost camera recordings. The Eulerian Video Magnification (EVM) technique was used to enhance the subtle differences in skin pixel intensity in the facial area. Two approaches were explored for performing regression: one using 12 novel features extracted from the amplified photoplethysmography (PPG) signal and Generalized Additive Models and a second using a 3D Convolution Neural Network (CNN) architecture on the raw amplified forehead video. The root mean square error in the estimated SpO2levels using both methods is minimal and in the accepted range for these applications.

The limited depth of field of optical lenses, makes multi-focus image fusion (MFIF) algorithms of vital importance. Lately, Convolutional Neural Networks (CNN) have been widely adopted in MFIF methods, however their predictions mostly lack structure and are limited by the size of the receptive field. Moreover, since images have noise due to various sources, the development of MFIF methods robust to image noise is required. A novel robust to noise Convolutional Neural Network-based Conditional Random Field (mf-CNNCRF) model is introduced. The model takes advantage of the powerful mapping between input and output of CNN networks and the long range interactions of the CRF models in order to reach structured inference. Rich priors for both unary and smoothness terms are learned by training CNN networks. The α -expansion graph-cut algorithm is used to reach structured inference for MFIF. A new dataset, which includes clean and noisy image pairs, is introduced and is used to train the networks of both CRF terms. A low-light MFIF dataset is also developed to demonstrate real-life noise introduced by the camera sensor. Qualitative and quantitative evaluation prove that mf-CNNCRF outperforms state-of-the-art MFIF methods for clean and noisy input images, while being more robust to different noise types without requiring prior knowledge of noise.

Managing random hardware faults requires the faults to be detected online, thus simplifying recovery. Algorithm-based fault tolerance has been proposed as a low-cost mechanism to check online the result of computations against random hardware failures. In this case, the checksum of the actual result is checked against a predicted checksum computed in parallel by a hardware checker. In this work, we target the design of such checkers for convolution engines that are currently the most critical building block in image processing and computer vision applications. The proposed convolution checksum checker, named ConvGuard, utilizes a newly introduced invariance condition of convolution to predictimplicitlythe output checksum using only the pixels at the border of the input image. In this way, ConvGuard reduces the power required for accumulating the input pixels without requiring large buffers to hold intermediate checksum results. The design of ConvGuard is generic and can be configured for different output sizes and strides. The experimental results show that ConvGuard utilizes only a small percentage of the area/power of an efficient convolution engine while being significantly smaller and more power efficient than a state-of-the-art checksum checker for various practical cases.

We explore the capabilities of a new biometric trait, which is based on information extracted through facial motion amplification. Unlike traditional facial biometric traits, the new biometric does not require the visibility of facial features, such as the eyes or nose, that are critical in common facial biometric algorithms. In this paper we propose the formation of a spatiotemporal facial blood flow map, constructed using small motion amplification. Experiments show that the proposed approach provides significant discriminatory capacity over different training and testing days and can be potentially used in situations where traditional facial biometrics may not be applicable.

The audio source separation problem is a well-known problem that was addressed using a variety of techniques. A common setback in these techniques is that the total number of sound sources in the audio mixture must be known beforehand. However, this knowledge is not always available and thus needs to be estimated. Many approaches have attempted to estimate the number of sources in an audio mixture. There are several clustering techniques that can count the sources in an audio mixture, nonetheless, there are cases, where the directionality of the audio data in the mixture may lead these techniques to failure. In this article, we propose a generalised Directional Fuzzy C-Means (DFCM) framework that offers a complete multi-dimensional, directional solution to this problem. Our proposal shows remarkably high performance in estimating the correct number of sources in the majority of the cases and in addition, it can be used as an effective mechanism to separate the sources. The complete source counting-separation framework can act as a robust low-complexity simultaneous solution to both problems.

We propose a new biometric trait based on facial motion amplification. The main advantage of the new biometric characteristic is that it does not rely on the visibility of critical facial features, such as nose, mouth, iris, or eyebrows. This makes it effective even when the respective areas are covered. Using the proposed system, facial image sequences are captured using an ordinary video camera and facial blood flow is calculated by means of small motion amplification. The calculated blood flow is captured from limited facial areas and is represented as a template that is suitable for identification purposes. Experiments on a new database show promising performance of the proposed approach, and provide evidence of the discriminatory capacity of the proposed biometric.

In this paper, a novel multi-focus image fusion algorithm based on conditional random field optimization (mf-CRF) is proposed. It is based on an unary term that includes the combined activity estimation of both high and low frequencies of the input images, while a spatially varying smoothness term is introduced, in order to align the graph-cut solution with boundaries of focused and defocused pixels. The proposed model retains the advantages of both spatial-domain methods and multi-spectral methods and by solving an energy minimization problem and finds an optimal solution for the multi-focus image fusion problem. Experimental results demonstrate the effectiveness of the proposed method that outperforms current state-of-the-art multi-focus image fusion algorithms in both qualitative and quantitative comparisons. In this paper, the successful application of the mf-CRF model in multi-modal image fusion (visible-infrared and medical) is also presented.

Document Image Binarization refers to the task of transforming a scanned image of a handwritten or printed document into a bi-level representation containing only characters and background. Here, we address the historic document image binarization problem using a three-stage methodology. Firstly, we remove possible stains and noise from the document image by estimating the document background image. The remaining background and character pixels are separated using a Local Co-occurrence Mapping, local contrast and a two-state Gaussian Mixture Model. In the last stage, possible isolated misclassified blobs are removed by a morphology operator. The proposed scheme offers robust and fast performance, especially for handwritten documents.

In this work, the 3D flow estimation problem is formulated in the 4D spatiotemporal frequency domain, and it is shown that 3D motion manifests itself as energy concentration along hyper-planes in that domain. Based on this, the construction and use of appropriate directional multidimensional “steerable” filters, which can extract directional energy in spacetime, is proposed. Steerable filters have been constructed for up to 3 dimensions. We extend the relevant mathematical definitions to multiple dimensions and formulate filter-based algorithms for 3D flow estimation. Experimental results on simulated and real data verify the efficiency of the algorithms.

In this paper, the authors exploit a multispectral image representation to perform more accurate document image binarisation compared to previous color representations. In the first stage, image fusion is employed to create a “document” and a “background” image. In the second stage, the FastICA algorithm is used to perform background subtraction. In the third stage, a spatial kernel K-harmonic means classifier binarizes the FastICA output. The proposed system outperforms previous efforts on document image binarization.

Directional or Circular statistics are pertaining to the analysis and interpretation of directions or rotations. In this work, a novel probability distribution is proposed to model multidimensional sparse directional data. The Generalized Directional Laplacian Distribution (DLD) is a hybrid between the Laplacian distribution and the von Mises-Fisher distribution. The distribution's parameters are estimated using Maximum-Likelihood Estimation over a set of training data points. Mixtures of Directional Laplacian Distributions (MDLD) are also introduced in order to model multiple concentrations of sparse directional data. The author explores the application of the derived DLD mixture model to cluster sound sources that exist in an underdetermined instantaneous sound mixture. The proposed model can solve the general${K\times L~(K< L)}$underdetermined instantaneous source separation problem, offering a fast and stable solution.

The aim of this paper is to provide an algorithm for image fusion which combines the techniques of Chebyshev polynomial (CP) approximation and independent component analysis (ICA), based on the regional information of input images. We present a region-based method that combines the merits of both techniques. It utilises segmentation to identify edges, texture and other important features in the input image and subsequently apply the different fusion methods according to regions. The proposed method exhibits better perceptual performance than individual CP and ICA fusion approaches especially in noise corrupted images.

This report documents in detail the research carried out by the author throughout his first year. The paper presents a novel method for fusing images in a domain concerning multiple sensors and modalities. Using Chebyshev polynomials as basis functions, the image is decomposed to perform fusion at feature level. Results show favourable performance compared to previous efforts on image fusion, namely ICA and DT-CWT, in noise affected images. The work presented here aims at providing a novel framework for future studies in image analysis and may introduce innovations in the fields of surveillance, medical imaging and remote sensing.

In this paper, the problem of moment-based shape orientation and symmetry classification is jointly considered. A generalization and modification of current state-of-the-art geometric moment-based functions is introduced. The properties of these functions are investigated thoroughly using Fourier series analysis and several observations and closed-form solutions are derived. We demonstrate the connection between the results presented in this work and symmetry detection principles suggested from previous complex moment-based formulations. The proposed analysis offers a unifying framework for shape orientation/symmetry detection. In the context of symmetry classification and matching, the second part of this work presents a frequency domain method, aiming at computing a robust moment-based feature set based on a true polar Fourier representation of image complex gradients and a novel periodicity detection scheme using subspace analysis. The proposed approach removes the requirement for accurate shape centroid estimation, which is the main limitation of moment-based methods, operating in the image spatial domain. The proposed framework demonstrated improved performance, compared to state-of-the-art methods.

The problem of shape-based recognition of objects under affine transformations is considered. We focus on the construction of a robust and highly discriminative affine invariant function that can be used for within-class object recognition applications. Using the boundaries of the objects of interest, a training scheme, based on principal component analysis (PCA), is proposed to derive a set of basis functions with desired properties. The derived bases are then used for the construction of a novel affine invariant function. The proposed invariant function is evaluated for the problem of aircraft silhouette identification and appears to achieve comparable performance to a popular wavelet-based affine invariant function. At the same time, the proposed framework is much simpler than that based on wavelet analysis.

The purpose of the Applied Multi-dimensional Fusion Project is to investigate the benefits that data fusion and related techniques may bring to future military Intelligence Surveillance Target Acquisition and Reconnaissance systems. In the course of this work, it is intended to show the practical application of some of the best multi-dimensional fusion research in the UK. This paper highlights the work done in the area of multi-spectral synthetic data generation, super-resolution, joint fusion and blind image restoration, multi-resolution target detection and identification and assessment measures for fusion. The paper also delves into the future aspirations of the work to look further at the use of hyper-spectral data and hyper-spectral fusion. The paper presents a wide work base in multi-dimensional fusion that is brought together through the use of common synthetic data, posing real-life problems faced in the theatre of war. Work done to date has produced practical pertinent research products with direct applicability to the problems posed.

Image fusion systems aim at transferring ‘interesting’ information from the input sensor images to the fused image. The common assumption for most fusion approaches is the existence of a high-quality reference image signal for all image parts in all input sensor images. In the case that there are common degraded areas in at least one of the input images, the fusion algorithms cannot improve the information provided there, but simply convey a combination of this degraded information to the output. The authors propose a combined spatial-domain method of fusion and restoration in order to identify these common degraded areas in the fused image and use a regularized restoration approach to enhance the content in these areas. The proposed approach was tested on both multi-focus and multi-modal image sets and produced interesting results.

The problem of blind separation of statistically independent sources from instantaneous mixtures, using the efficient framework of independent component analysis (ICA), has been widely addressed in the literature. In this letter, the authors propose a sequential blind signal extraction algorithm that attempts to identify smooth sources in instantaneous mixtures. The approach incorporates smoothness constraints in the traditional negentropy cost function to extract smooth components, using an approximate second-order optimization method

A scheme, based on principal component analysis (PCA), is proposed that can be used for the recognition of 2-D planar shapes under affine transformations. A PCA step is first used to map the object boundary to its canonical form, reducing the problem of the nonuniform sampling of the object contour introduced by the affine transformation. Then, a PCA-based scheme is employed to train a set of basis functions on the signals extracted from the objects' boundaries. The derived bases are used to analyze the boundary locally. Based on the theory of invariants and local boundary analysis, a novel invariant function is constructed. The performance of the proposed framework is compared with a standard wavelet-based approach with promising results.

In this paper, we explore the problem of sound source separation and identification from a two-sensor instantaneous mixture. The estimation of the mixing and the sources is performed using Laplacian mixture models (LMM). The proposed algorithm fits the model using batch processing of the observed data and performs separation using either a hard or a soft decision scheme. An extension of the algorithm to online source separation, where the samples are arriving in a real-time fashion, is also presented. The online version demonstrates several promising source separation possibilities in the case of nonstationary mixing.

Image fusion can be viewed as a process that incorporates essential information from different modality sensors into a composite image. The use of bases trained using independent component analysis (ICA) for image fusion has been highlighted recently. Common fusion rules can be used in the ICA fusion framework with promising results. In this paper, the authors propose an adaptive fusion scheme, based on the ICA fusion framework, that maximises the sparsity of the fusion image in the transform domain

The authors explore the use of Laplacian mixture models (LMMs) to address the overcomplete blind source separation problem in the case that the source signals are very sparse. A two-sensor setup was used to separate an instantaneous mixture of sources. A hard and a soft decision scheme were introduced to perform separation. The algorithm exhibits good performance as far as separation quality and convergence speed are concerned.

The problem of separation of audio sources recorded in a real world situation is well established in modern literature. A method to solve this problem is blind source separation (BSS) using independent component analysis (ICA). The recording environment is usually modeled as convolutive. Previous research on ICA of instantaneous mixtures provided solid background for the separation of convolved mixtures. The authors revise current approaches on the subject and propose a fast frequency domain ICA framework, providing a solution for the apparent permutation problem encountered in these methods.