Optical flow estimation is a classical yet challenging task in computer vision. One of the essential factors in accurately predicting optical flow is to alleviate occlusions between frames. However, it is still a thorny problem for current top-performing optical flow estimation methods due to insufficient local evidence to model occluded areas. In this paper, we propose the Super Kernel Flow Network (SKFlow), a CNN architecture to ameliorate the impacts of occlusions on optical flow estimation. SKFlow benefits from the super kernels which bring enlarged receptive fields to complement the absent matching information and recover the occluded motions. We present efficient super kernel designs by utilizing conical connections and hybrid depth-wise convolutions. Extensive experiments demonstrate the effectiveness of SKFlow on multiple benchmarks, especially in the occluded areas. Without pre-trained backbones on ImageNet and with a modest increase in computation, SKFlow achieves compelling performance and ranks $\textbf{1st}$ among currently published methods on the Sintel benchmark. On the challenging Sintel clean and final passes (test), SKFlow surpasses the best-published result in the unmatched areas ($7.96$ and $12.50$) by $9.09\%$ and $7.92\%$. The code is available at https://github.com/littlespray/SKFlow.

As a severe psychiatric disorder disease, depression is a state of low mood and aversion to activity, which prevents a person from functioning normally in both work and daily lives. The study on automated mental health assessment has been given increasing attentions in recent years. In this paper, we study the problem of automatic diagnosis of depression. A new approach to predict the Beck Depression Inventory II (BDI-II) values from video data is proposed based on the deep networks. The proposed framework is designed in a two stream manner, aiming at capturing both the facial appearance and dynamics. Further, we employ joint tuning layers that can implicitly integrate the appearance and dynamic information. Experiments are conducted on two depression databases, AVEC2013 and AVEC2014. The experimental results show that our proposed approach significantly improve the depression prediction performance, compared to other visual-based approaches.

Still-to-video (S2V) face recognition has recently attracted attention from researchers because of its great applications in real-world scenarios. In S2V FR, still images are usually of high quality, captured from cooperative users under controlled environment, such as mugshots, while video clips may be acquired with low resolutions and low quality, from non-cooperative users under uncontrolled environment. Because of those significant differences, we interpret the S2V FR as a heterogeneous matching problem, and propose an approach aiming at building multiple “bridges” between those two heterogeneous face modalities. Considering the unbalanced distributions and large diversities between two modalities, we propose to exploit a Grassmann manifold learning method to construct subspaces in between to find connections (or transitions) between the still images and video clips. Multiple geodesic flows are generated connecting the subspace of still images and the clustered subspace centers of videos, which are representative and robust to characterize the relationship between still images and video frames. Extensive experiments are conducted on two large scale benchmark databases, COX-S2V and PaSC, with different recognition tasks: face identification and verification. The experimental results show that the proposed approach outperforms the state-of-the-art methods under the same experimental settings.

Depression is a severe psychiatric disorder preventing a person from functioning normally in both work and daily lives. Currently, diagnosis of depression requires extensive participation from clinical experts. It has drawn much attention to develop an automatic system for efficient and reliable diagnosis of depression. Under the influence of depression, visual-based behavior disorder is readily observable. This paper presents a novel method of exploring facial region visual-based nonverbal behavior analysis for automatic depression diagnosis. Dynamic feature descriptors are extracted from facial region subvolumes, and sparse coding is employed to implicitly organize the extracted feature descriptors for depression diagnosis. Discriminative mapping and decision fusion are applied to further improve the accuracy of visual-based diagnosis. The integrated approach has been tested on the AVEC2013 depression database and the best visual-based mean absolute error/root mean square error results have been achieved.

Human action recognition is a very active research topic in computer vision and pattern recognition. Recently, it has shown a great potential for human action recognition using the three-dimensional (3D) depth data captured by the emerging RGB-D sensors. Several features and/or algorithms have been proposed for depth-based action recognition. A question is raised: Can we find some complementary features and combine them to improve the accuracy significantly for depth-based action recognition? To address the question and have a better understanding of the problem, we study the fusion of different features for depth-based action recognition. Although data fusion has shown great success in other areas, it has not been well studied yet on 3D action recognition. Some issues need to be addressed, for example, whether the fusion is helpful or not for depth-based action recognition, and how to do the fusion properly. In this article, we study different fusion schemes comprehensively, using diverse features for action characterization in depth videos. Two different levels of fusion schemes are investigated, that is, feature level and decision level. Various methods are explored at each fusion level. Four different features are considered to characterize the depth action patterns from different aspects. The experiments are conducted on four challenging depth action databases, in order to evaluate and find the best fusion methods generally. Our experimental results show that the four different features investigated in the article can complement each other, and appropriate fusion methods can improve the recognition accuracies significantly over each individual feature. More importantly, our fusion-based action recognition outperforms the state-of-the-art approaches on these challenging databases.

Human action recognition is important in image and video processing with many applications. With the development of sensor technology, different cameras can be used for action acquisition, e.g., infrared cameras. Is it possible to adapt the visible light action recognizers to a new modality or domain? In this paper, we study the feasibility to adapt the action recognizer learned from visible light spectrum to infrared. A preliminary result is obtained on a large database based on an adaptive learning method, demonstrating the potential to perform cross-spectral action recognition.