Self-supervised pretraining has been proved to be effective in learning transferable representations that benefit various visual tasks. This paper asks this question: can self-supervised pretraining learn general facial representations for various facial analysis tasks? Recent efforts to-ward this goal are limited to treating each face image as a whole, i.e., learning consistent facial representations at the image-level, which overlooks the “consistency of local facial representations“(i.e., facial regions like eyes, nose, etc). In this work, we make a first attempt to propose a novel self-supervised facial representation learning framework to learn consistent global and local facial representations, Facial Region Awareness (FRA). Specifically, we explicitly enforce the consistency of facial regions by matching the local facial representations across views, which are extracted with learned heatmaps highlighting the facial regions. Inspired by the mask prediction in supervised semantic segmentation, we obtain the heatmaps via cosine simi-larity between the per-pixel projection of feature maps and “facial mask embeddings” computed from learnable positional embeddings, which leverage the attention mechanism to globally look up the facial image for facial regions. To learn such heatmaps, we formulate the learning of facial mask embeddings as a deep clustering problem by assigning the pixel features from the feature maps to them. The transfer learning results on facial classification and regression tasks show that our FRA outperforms previous pretrained models and more importantly, using ResNet as the unified backbone for various tasks, our FRA achieves comparable or even better performance compared with SOTA methods in facial analysis tasks.

Whilst contrastive learning yields powerful representations by matching different augmented views of the same instance, it lacks the ability to capture the similarities between different instances. One popular way to address this limitation is by learning global features (after the global pooling) to capture inter-instance relationships based on knowledge distillation, where the global features of the teacher are used to guide the learning of the global features of the student. Inspired by cross-modality learning, we extend this existing framework that only learns from global features by encouraging the global features and intermediate layer features to learn from each other. This leads to our novel self-supervised framework: cross-context learning between global and hypercolumn features (CGH), that enforces the consistency of instance relations between low-and high-level semantics. Specifically, we stack the intermediate feature maps to construct a "hypercolumn" representation so that we can measure instance relations using two contexts (hypercolumn and global feature) separately, and then use the relations of one context to guide the learning of the other. This cross-context learning allows the model to learn from the differences between the two contexts. The experimental results on linear classification and downstream tasks show that our method outperforms the state-of-the-art methods.

Fifth generation with high dimensions and capabilities is expected to fulfil the requirements of the Tactile Internet. Tracking provides strong support for intuitive interaction with interfaces by hands, eyes, bodies, etc. Such interfaces can be used in the Tactile Internet for interaction with real and virtual objects. The trackers based on tracking-by-detection framework rely on manual feature detectors for robust tracking, which is particularly useful for specific objects like humans but cannot handle generic tracking problems. Therefore, a sparse feature learning method beyond manual design is proposed to learn features from the samples sampled during tracking. The basic idea is to learn a dictionary from the samples in the previous frames and construct feature representations to represent the object for detection of the location in the current frame. The samples are patches centered at the keypoints based on an adaptive features from accelerated segment test (FAST) detector with local threshold. The dictionary is learned with sparse coding for sparse representations and atoms of the dictionary are grouped to describe the local orientation of these samples. The integrated features are built after rectification of the sparse representations. The correlation filter is used to infer the object location from the sparse features. The qualitative and quantitative experimental results on OTB100 show the advantage of the proposed tracker against current state-of-the-art trackers in terms of accuracy.