The low resolution (LR) problem is rather challenging in face analysis. Most existing face hallucination methods assume that LR face images have only one resolution, but multiple resolutions may be available from different sources. To solve this issue, we propose a novel simultaneous facial canonical correlation representation learning method for face hallucination, which seeks latent correlation subspaces for multi-resolution views. Our method jointly solves multiple linear transformations by optimizing a correlation summation criterion of all pairs of resolutions. The neighborhood reconstruction is used to infer the HR facial canonical correlation representation of LR face inputs. Extensive experimental results show the superiority of our proposed method in terms of quantitative and qualitative evaluations.

The recently proposed Bayesian Flow Networks (BFNs) show great potential in modeling parameter spaces via a diffusion process, offering a unified strategy for handling continuous, discrete data. However, these parameter diffusion models cannot learn high-level semantic representation from the parameter space since common encoders, which encode data into one static representation, can- not capture semantic changes in parameters. This motivates a new direction: learning semantic representations hidden in the param- eter spaces to characterize noisy data. Accordingly, we propose a representation learning framework named SepDiff which operates in the parameter space to obtain parameter-wise latent semantics that exhibit progressive structures. Specifically, SepDiff proposes a self-encoder to learn latent semantics directly from parameters, rather than from observations. The encoder is then integrated into parameter diffusion model, enabling representation learning with various formats of observations. Mutual information terms further promote the disentanglement of latent semantics and capture mean- ingful semantics simultaneously. We illustrate seven representation learning tasks in SepDiff via expanding this parameter diffusion model, and extensive quantitative experimental results demonstrate the superior effectiveness of SepDiff in learning parameter repre- sentation.

Canonical correlation analysis (CCA) matters in multi-view representation learning. But, CCA and its most variants are essentially based on explicit or implicit covariance matrices. It means that they have no ability to model the nonlinear relationship among features due to intrinsic linearity of covariance. In this paper, we address the preceding problem and propose a novel canonical F-correlation framework by exploring and exploiting the nonlinear relationship between different features. The framework projects each feature rather than observation into a certain new space by an arbitrary nonlinear mapping, thus resulting in more flexibility in real applications. With this frame-work as a tool, we propose a correlative covariation projection (CCP) method by using an explicit nonlinear mapping. Moreover, we further propose a multiset version of CCP dubbed MCCP for learning compact representation of more than two views. The proposed MCCP is solved by an iterative method, and we prove the convergence of this iteration. A series of experimental results on six benchmark datasets demonstrate the effectiveness of our proposed CCP and MCCP methods.

Component analysis (CA) is a powerful technique for learning discriminative representations in various computer vision tasks. Typical CA methods are essentially based on the covariance matrix of training data. But, the covariance matrix has obvious disadvantages such as failing to model complex relationship among features and singularity in small sample size cases. In this letter, we propose a general covariance measure to achieve better data representations. The proposed covariance is characterized by a nonlinear mapping determined by domain-specific applications, thus leading to more advantages, flexibility, and applicability in practice. With general covariance, we further present two novel CA methods for learning compact representations and discuss their differences from conventional methods. A series of experimental results on nine benchmark data sets demonstrate the effectiveness of the proposed methods in terms of accuracy.

Face hallucination (FH) is an effective technique for super-resolving low-resolution (LR) face images. In real-world applications, a face image usually has multiple distinct low resolutions. Most existing FH methods can not effectively deal with multiple LR views simultaneously. To solve this issue, we present a multi-set partial least squares (MPLS) approach and its kernel extension for jointly learning the nonlinear consistency of multi-resolution facial features. With nonlinear MPLS, we present a novel simultaneous super-resolution coherent facial feature method for the face images with multiple LRs, which has capacity of jointly learning the nonlinear relationships between multiple facial resolutions. Experimental results demonstrate the effectiveness and robustness of our proposed FH method.

Face super-resolution (FSR) is an effective way to solve low-resolution (LR) problems in face analysis. But, most FSR methods only consider that LR face images have a single resolution, which is usually not consistent with practical situations due to the existence of multiple resolutions. To date, simultaneously learning the mappings from multiple LRs to high resolution (HR) has not been given proper attention. To solve this issue, we first propose a multi-set partial least squares (MPLS) approach to jointly deal with multi-set random variables via a recursive optimization. With MPLS, we then present a novel FSR method called MPLS-FH to simultaneously learn multiple resolution-specific mappings for various LR views from the same source. Concretely, MPLS-FH first divides multi-resolution face images into many patches. Then, it jointly learns the latent coherent features of principal-component embeddings of multi-resolution patches. Last, it super-resolves the input LR face by cross-resolution neighborhood search. Experimental results demonstrate the effectiveness of the proposed method in terms of quantitative and qualitative evaluations.