Learning a decent representation from unlabeled time series is a challenging task, especially when the time series data is derived from diverse channels at different sampling rates. Our motivation stems from the financial domain, where sparsely labeled covariates are commonly collected at different frequencies, *e.g.*, daily stock market index, monthly unemployment rate and quarterly net revenue of a certain listed corporation. This paper presents **M**ulti-**F**requency **C**ontrastive **L**earning **R**epresentation (MF-CLR), aimed at learning a good representation of multi-frequency time series in a self-supervised paradigm by leveraging the ability of contrastive learning. MF-CLR introduces a hierarchical mechanism that spans across different frequencies along the feature dimension. Within each contrastive block, two groups of subseries with adjacent frequencies are embedded based on our proposed cross-frequency consistency. To validate the effectiveness of MF-CLR, we conduct extensive experiments on five downstream tasks, including long-term and short-term forecasting, classification, anomaly detection and imputation. Experimental evidence shows that MF-CLR delivers a leading performance in all the downstream tasks and keeps consistent performance across different target dataset scales in the transfer learning scenario.

The deep neural networks (DNNs) trained by adversarial training (AT) usually suffered from significant robust generalization gap, i.e., DNNs achieve high training robustness but low test robustness. In this paper, we propose a generic method to boost the robust generalization of AT methods from the novel perspective of attribution span. To this end, compared with standard DNNs, we discover that the generalization gap of adversarially trained DNNs is caused by the smaller attribution span on the input image. In other words, adversarially trained DNNs tend to focus on specific visual concepts on training images, causing its limitation on test robustness. In this way, to enhance the robustness, we propose an effective method to enlarge the learned attribution span. Besides, we use hybrid feature statistics for feature fusion to enrich the diversity of features. Extensive experiments show that our method can effectively improves robustness of adversarially trained DNNs, outperforming previous SOTA methods. Furthermore, we provide a theoretical analysis of our method to prove its effectiveness.

Non-parallel voice conversion (VC) is a technique of transfer-ring voice from one style to another without using a parallel corpus in model training. Various methods are proposed to approach non-parallel VC using deep neural networks. Among them, CycleGAN-VC and its variants have been widely accepted as benchmark methods. However, there is still a gap to bridge between the real target and converted voice and an increased number of parameters leads to slow convergence in training process. Inspired by recent advancements in unsupervised image translation, we propose a new end-to-end unsupervised framework U-GAT-VC that adopts a novel inter- and intra-attention mechanism to guide the voice conversion to focus on more important regions in spectrograms. We also introduce disentangle perceptual loss in our model to capture high-level spectral features. Subjective and objective evaluations shows our proposed model outperforms CycleGAN-VC2/3 in terms of conversion quality and voice naturalness.

Domain adaptation (DA) tries to tackle the scenarios when the test data does not fully follow the same distribution of the training data, and multi-source domain adaptation (MSDA) is very attractive for real world applications. By learning from large-scale unlabeled samples, self-supervised learning has now become a new trend in deep learning. It is worth noting that both self-supervised learning and multi-source domain adaptation share a similar goal: they both aim to leverage unlabeled data to learn more expressive representations. Unfortunately, traditional multi-task self-supervised learning faces two challenges: (1) the pretext task may not strongly relate to the downstream task, thus it could be difficult to learn useful knowledge being shared from the pretext task to the target task; (2) when the same feature extractor is shared between the pretext task and the downstream one and only different prediction heads are used, it is ineffective to enable inter-task information exchange and knowledge sharing. To address these issues, we propose a novel Self-Supervised Graph Neural Network (SSG), where a graph neural network is used as the bridge to enable more effective inter-task information exchange and knowledge sharing. More expressive representation is learned by adopting a mask token strategy to mask some domain information. Our extensive experiments have demonstrated that our proposed SSG method has achieved state-of-the-art results over four multi-source domain adaptation datasets, which have shown the effectiveness of our proposed SSG method from different aspects.

While deep learning makes significant achievements in Artificial Intelligence (AI), the lack of transparency has limited its broad application in various vertical domains. Explainability is not only a gateway between AI and society, but also a powerful feature to detect flaws of the models and bias of the data. Local Interpretable Model-agnostic Explanation (LIME) is a widely-accepted technique that explains the predictions of any classifier faithfully by learning an interpretable model locally around the predicted instance. However, the sampling operation in the standard implementation of LIME is defective. Perturbed samples are generated from an uniform distribution, ignoring the complicated correlation between features. Moreover, as the local decision boundary is non-linear for most complex networks, linear approximation may produce serious errors. This paper proposes an high-interpretability and high-fidelity local explanation method, known as Kernel-based LIME with Feature Dependency Sampling (KLFDS). KLFDS enhances interpretability by feature sampling with intrinsic dependency. Besides, KLFDS improves the local explanation fidelity by approximating nonlinear boundary of local decision. We evaluate our method on image classification tasks and results show that KLFDS's explanation of the black-box model achieves much better performance than original LIME.

Facial landmark detection is fundamental for various face-related applications such as interactive avatars on mobile devices. Awareness of detection failure is critical for practical applications because even occasional failures to detect facial landmarks lead to bad user experience. This letter proposes a fast and robust AdaBoost Based Cascade Detector (ABCD) for discerning failures from shape regression in video on mobile devices. A vector of randomly sampled pixel intensities near facial landmarks is taken as the input feature for AdaBoost classifiers. Several AdaBoost classifiers are cascaded together for robustness, computational efficiency and to augment the theoretical number of false samples in training. With this failure detector, the correctly estimated shape of the previous frame can be utilized in the next frame for initialization, which not only improves the regression accuracy but also saves on face searching time. ABCD is incorporated into a recently proposed facial landmark detection algorithm Face Alignment by Explicit Shape Regression (FAESR). Experiments on videos show that failure awareness powered FAESR yields an accurate and automatic facial landmark detection with very low computational costs, which is suitable for real time application on mobile devices.

Cosmetic makeup is a general event in our daily life, which improves women's beauties and attractions. But, it is difficult for ordinary users to make a wonderful makeup as the cover girls. Moreover, when you are in nude look and want to share better look with your friends, the fastest and easiest way is virtual makeup. However, current existing makeup software needs many user inputs to adjust face landmarks, which influence the user experience. And, it cannot remove the flaws on skin as good as the real cosmetic makeup. In this paper, we describe an automatic framework to apply a cosmetic makeup and skin beautification to your face, which can be selected from many example make-up face images. Our method detects the face landmarks with existing algorithm and adjusts the landmark with skin color Gaussian Mixture model based segmentation. Then, the skin color area is separated into three layers, and makeup is transferred with different method to different layers. The results look pretty good for some natural input face images.

Along with the inclusion of GPU cores within the same CPU die, the performance of Intel's processor-graphics has been significantly improved over earlier generation of integrated graphics. The need to efficiently harness the computational power of the GPU in the same CPU die is more than ever. This paper presents a highly optimized Haar-based face detector which efficiently exploits both CPU and GPU computing power on the latest Sandy Bridge processor. The classification procedure of Haar-based cascade detector is partitioned to two phases in order to leverage both thread level and data level parallelism in the GPU. The image downscaling and integral image calculation running in the CPU core can work with the GPU in parallel. Compared to CPU-alone implementation, the experiments show that our proposed GPU accelerated implementation achieves a 3.07x speedup with more than 50% power reduction on the latest Sandy Bridge processor. On the other hand, our implementation is also more efficient than the CUDA implementation on the NVidia GT430 card in terms of performance as well as power. In addition, our proposed method presents a general approach for task partitioning between CPU and GPU, thus being beneficial not only for face detection but also for other multimedia and computer vision techniques.

Facial landmark detection is an essential module in many face related applications and it often appears as the most time consuming part in face processing pipeline. This paper proposes a fast and effective method for facial landmark detection using Haar cascade classifiers and a simple 3D head model, which not only detects the position of landmark points but also gives an estimation of head pose such as yaw and pitch angles. To reduce the amount of computation, only 7 landmark points are detected (4 eye corners, 2 mouth corners, 1 nose tip) that generally meets the requirement of face alignment and face recognition. Experiment on multiple datasets shows our algorithm can provide sufficient accuracy of facial landmark localization while being able to run in super real-time at Intel Atom 1.3 GHz embedded processors.

Avatar is the virtual representation of user's facial, body, and motion characteristics in computer game, social network, and augmented reality. Facial modeling needs enormous efforts to achieve immersive experience in applications like avatar chatting or online makeover. Great challenge exists in robust detection of 2D facial prominent points and mapping them to 3D models in a parameterized manner. Another challenge is how to characterize semantic components of eyes, mouth, nose, and cheek rather than low level mesh geometries. In this paper, we proposed an augmented makeover framework to deal with aforementioned challenges. Aiming to provide amateurs with flexible customizations, morphable model is constructed from a set of scanned 3D face data set. Appearance personalization is carried out in the offline phase where single image and multiple views are discussed respectively to generate deformative shape in a progressive manner. Augmentation is implemented in the online phase where a fast and robust 3D tracking is used to balance the tradeoff between accuracy and real-time requirements. By this means, immersive Human Computer Interaction such as virtual makeover and photo-realistic avatar chatting could be achieved.

This article pursues a two-fold goal. First we introduce degree of goal directedness (DGD), a novel quantitative dimension for the taxonomy of navigation tasks in general. As an attempt to operationalize the DGD concept in the context of electronic documents navigation, we introduce the serial target-acquisition (STA) experimental paradigm. We suggest that DGD and the STA paradigm may usefully enrich the conceptual toolkit of HCI research for the evaluation of navigation techniques. Our second goal is to illustrate the utility of the DGD concept by showing with a concrete example, Perspective Drag, the refinement it allows in evaluating navigation techniques. We report data obtained from two experiments with the STA paradigm that cast light on what Perspective Drag is specifically good for: it is particularly suitable in realistic task contexts where navigation is less than 100% directed by its terminal goal, that is, where the user wants not only to reach a particular item but also to pick up information from the document during document traversal.

Touchstone is an open-source experiment design platform designed to help establish a solid research foundation for HCI in the area of novel interaction techniques. Touchstone includes a design platform for exploring alternative designs of controlled laboratory experiments, a run platform for running subjects and a limited analysis platform for advice and access to on-line statistics packages. Designed for HCI researchers and their students, Touchstone facilitates the process of creating new experiments, as well as replicating and extending experiments in the research literature. We tested Touchstone by designing two controlled experiments. One illustrates how to create a new experiment from scratch. The other replicates and extends a previous study of multiscale pointing interaction techniques: OrthoZoom was fastest, followed by bi-manual Pan & Zoom; SDAZ and traditional Pan & Zoom were consistently slower.

For video summarization and retrieval, one of the important modules is to group temporal-spatial coherent shots into high-level semantic video clips namely scene segmentation. In this paper, we propose a novel scene segmentation and categorization approach using normalized graph cuts(NCuts). Starting from a set of shots, we first calculate shot similarity from shot key frames. Then by modeling scene segmentation as a graph partition problem where each node is a shot and the weight of edge represents the similarity between two shots, we employ NCuts to find the optimal scene segmentation and automatically decide the optimum scene number by Q function. To discover more useful information from scenes, we analyze the temporal layout patterns of shots, and automatically categorize scenes into two different types, i.e. parallel event scenes and serial event scenes. Extensive experiments are tested on movie, and TV series. The promising results demonstrate that the proposed NCuts based scene segmentation and categorization methods are effective in practice.

The current GUI is like a flight simulator whose camera points fixedly at right angle to the document, thus preventing users from looking ahead while navigating. We argue that perspective viewing of usual planar documents can help navigation. We analyze the scale implosion problem that arises with tilted cameras and we report the data of a formal experiment on document navigation with perspective views.

In many vision problems, the appearances of the observed images, e.g. the human facial images, are often influenced by multiple underlying factors. In this paper, a kernel-based factorization framework is proposed to analyze a multifactor dataset. Specifically, we perform N-mode singular value decomposition (N-mode SVD) in a higher dimensional feature space instead of the input space by using kernel approaches. Given an input sample, its specific underlying factors which may be all absent in the training set can be extracted and translated from one sample to another by using kernel-based 'translation'. Therefore our framework is suitable for tasks of new image synthesis and underlying factor recognition. We demonstrate the capabilities of our framework on ensembles of facial images subjected to different person identities, viewpoints and illuminations with high-quality synthetic faces and high face recognition accuracy.

Facial images captured by camera are usually influenced by several different kinds of factor, such as human identity, facial expression, and etc. Therefore facial expressive image analysis plays an important role in many different applications, such as expression recognition, face recognition with expressive images, facial expressive image synthesis and etc. In this paper, "human identity" and "facial expression" are regarded as two influence factors of face appearance, and a novel kernel-based bilinear factorization method is proposed to decouple the interaction from each other. The trained non-linear factorization model has been used for facial expressive image translation, and to person and/or expression recognition, and got promising results. In this paper, the principle of our method is described and some preliminary experimental results are included.