Diffusion models have revolutionized generative modeling, enabling unprecedented realism in image and video synthesis. This success has sparked interest in leveraging their representations for visual understanding tasks. While recent works have explored this potential for image generation, the visual understanding capabilities of video diffusion models remain largely uncharted. To address this gap, we systematically compare the same model architecture trained for video versus image generation, analyzing the performance of their latent representations on various downstream tasks including image classification, action recognition, depth estimation, and tracking. Results show that video diffusion models consistently outperform their image counterparts, though we find a striking range in the extent of this superiority. We further analyze features extracted from different layers and with varying noise levels, as well as the effect of model size and training budget on representation and generation quality. This work marks the first direct comparison of video and image diffusion objectives for visual understanding, offering insights into the role of temporal information in representation learning.

Even though convolutional neural networks have become the method of choice in many fields of computer vision, they still lack interpretability and are usually designed manually in a cumbersome trial-and-error process. This paper aims at overcoming those limitations by proposing a deep neural network, which is designed in a systematic fashion and is interpretable, by integrating multiresolution analysis at the core of the deep neural network design. By using the lifting scheme, it is possible to generate a wavelet representation and design a network capable of learning wavelet coefficients in an end-to-end form. Compared to state-of-the-art architectures, the proposed model requires less hyper-parameter tuning and achieves competitive accuracy in image classification tasks. The Code implemented for this research is available at https://github.com/mxbastidasr/DAWN_WACV2020.

A graph neural network (GNN) for image understanding based on multiple cues is proposed in this paper. Compared to traditional feature and decision fusion approaches that neglect the fact that features can interact and exchange information, the proposed GNN is able to pass information among features extracted from different models. Two image understanding tasks, namely group-level emotion recognition (GER) and event recognition, which are highly semantic and require the interaction of several deep models to synthesize multiple cues, were selected to validate the performance of the proposed method. It is shown through experiments that the proposed method achieves state-of-the-art performance on the selected image understanding tasks. In addition, a new group-level emotion recognition database is introduced and shared in this paper.

Social relations reveal the interpersonal association of human beings. Developing techniques to automatically recognize social relations from visual data has great potential for improving human-computer interaction. In this paper, a hybrid deep network is proposed to predict the social relations between two human beings in an image. Unlike existing methods that typically learn deep learning models from scratch, a VGG-FACE model previously trained for face recognition is fine-tuned on a social relation database and used as branches of a siamese-like network. Moreover, a deep network is proposed to extract scene features that contain high-level information related to social relations from whole images and its predictions are fused with the predictions of the siamese network to generate the final result. Experiments show that the proposed approach saves the effort of pre-training and preparing auxiliary datasets, i.e. facial attribute datasets, and outperforms state-of-the-art methods.

This paper addresses the problem of generating recommendations for completing the outfit given that a user is interested in a particular apparel item. The proposed method is based on a siamese network used for feature extraction followed by a fully-connected network used for learning a fashion compatibility metric. The embeddings generated by the siamese network are augmented with color histogram features motivated by the important role that color plays in determining fashion compatibility. The training of the network is formulated as a maximum a posteriori (MAP) problem where Laplacian distributions are assumed for the filters of the siamese network to promote sparsity and matrix-variate normal distributions are assumed for the weights of the metric network to efficiently exploit correlations between the input units of each fully-connected layer.

Ordinal regression aims to classify instances into ordinal categories. In this paper, body mass index (BMI) category estimation from facial images is cast as an ordinal regression problem. In particular, noisy binary search algorithms based on pairwise comparisons are employed to exploit the ordinal relationship among BMI categories. Comparisons are performed with Siamese architectures, one of which uses the Bradley-Terry model probabilities as target. The Bradley-Terry model describes probabilities of the possible outcomes when elements of a set are repeatedly compared with one another in pairs. Experimental results show that our approach outperforms classification and regression-based methods at estimating BMI categories.

Smile detection from unconstrained facial images is a specialized and challenging problem. As one of the most informative expressions, smiles convey basic underlying emotions, such as happiness and satisfaction, and leads to multiple applications, such as human behavior analysis and interactive controlling. Compared to the size of databases for face recognition, far less labeled data is available for training smile detection systems. This paper proposes an efficient transfer learning-based smile detection approach to leverage the large amount of labeled data from face recognition datasets and to alleviate overfitting on smile detection. A well-trained deep face recognition model is explored and fine-tuned for smile detection in the wild, unlike previous works which use either hand-engineered features or train deep convolutional networks from scratch. Three different models are built as a result of fine-tuning the face recognition model with different inputs, including aligned, unaligned and grayscale images generated from the GENKI-4K dataset. Experiments show that the proposed approach achieves improved state-of-the-art performance. Robustness of the model to noise and blur artifacts is also evaluated in this paper.

This paper presents a hybrid deep learning network submitted to the 6th Emotion Recognition in the Wild (EmotiW 2018) Grand Challenge [9], in the category of group-level emotion recognition. Advanced deep learning models trained individually on faces, scenes, skeletons and salient regions using visual attention mechanisms are fused to classify the emotion of a group of people in an image as positive, neutral or negative. Experimental results show that the proposed hybrid network achieves 78.98% and 68.08% classification accuracy on the validation and testing sets, respectively. These results outperform the baseline of 64% and 61%, and achieved the first place in the challenge.

This paper presents the work submitted to the Group-level Emotion Recognition sub-challenge, which is part of the 5th Emotion Recognition in the Wild (EmotiW 2017) Challenge. The task of this sub-challenge is to classify the emotion of a group of people in each image as positive, neutral or negative. To address this task, a hybrid network that incorporates global scene features, skeleton features of the group, and local facial features is developed. Specifically, deep convolutional neural networks (CNNs) are first trained on the faces of the group, the whole images and the skeletons of the group, and then fused to perform the group-level emotion prediction. Experimental results show that the proposed network achieves 80.05% and 80.61% on the validation and testing sets, respectively, outperforming the baseline of 52.97% and 53.62%.

Recent results in telecardiology show that compressed sensing (CS) is a promising tool to lower energy consumption in wireless body area networks for electrocardiogram (ECG) monitoring. However, the performance of current CS-based algorithms, in terms of compression rate and reconstruction quality of the ECG, still falls short of the performance attained by state-of-the-art wavelet-based algorithms. In this paper, we propose to exploit the structure of the wavelet representation of the ECG signal to boost the performance of CS-based methods for compression and reconstruction of ECG signals. More precisely, we incorporate prior information about the wavelet dependencies across scales into the reconstruction algorithms and exploit the high fraction of common support of the wavelet coefficients of consecutive ECG segments. Experimental results utilizing the MIT-BIH Arrhythmia Database show that significant performance gains, in terms of compression rate and reconstruction quality, can be obtained by the proposed algorithms compared to current CS-based methods.

Compressive sensing has recently been applied to electrocardiogram (ECG) acquisition and reconstruction with the aim of lowering energy consumption and sampling rates in wireless body area networks for ambulatory ECG monitoring. However, most current methods only adopt a sparse prior on the ECG wavelet representation. In this paper, we propose to further exploit the wavelet representation structure by incorporating two properties in the formulation of the optimization problem: the exponentially decaying magnitude of the detail coefficients across scales and the accumulation of signal energy in the approximation subband. We derive a weighted ℓ1minimization algorithm, based on a maximum a posteriori (MAP) approach, that leads to a significant reduction in the number of measurements and superior reconstruction performance compared to current CS-based methods with application to wireless ECG systems.

Recent works in modified compressed sensing (CS) show that reconstruction of sparse or compressible signals with partially known support yields better results than traditional CS. In this paper, we extend the ideas of these works to modify the iterative hard thresholding (IHT) algorithm to incorporate known support in the recovery process. We present a theoretical analysis that shows that including prior support information relaxes the conditions for stable reconstruction. Numerical results show that the IHT modification improves performance, thereby needing fewer samples to yield an approximate reconstruction.

Compressive sensing (CS) is a new approach for the acquisition and recovery of sparse signals that enables sampling rates significantly below the classical Nyquist rate. Based on the fact that electrocardiogram (ECG) signals can be approximated by a linear combination of a few coefficients taken from a Wavelet basis, we propose a compressed sensing-based approach for ECG signal compression. ECG signals generally show redundancy between adjacent heartbeats due to its quasi-periodic structure. We show that this redundancy implies a high fraction of common support between consecutive heartbeats. The contribution of this paper lies in the use of distributed compressed sensing to exploit the common sup port between samples of jointly sparse adjacent beats. Simulation results suggest that compressed sensing should be considered as a plausible methodology for ECG compression.

Recent works in modified compressed sensing (CS) show that reconstruction of sparse or compressible signals with partially known support yields better results than traditional CS. In this paper, we extend the ideas of these works to modify three iterative algorithms to incorporate the known support in the recovery process. The performance and effect of the prior information are studied through simulations. Results show that the modification of iterative algorithms improves their performance, needing fewer samples to yield an approximate reconstruction.