Existing face anti-spoofing (FAS) methods have primarily focused on close-set or cross-domain settings with a few pre-defined presentation attacks (PAs). However, with the continual emergence of diverse PAs, we argue that developing a generalizable FAS detector for unseen PAs deserves more attention from the FAS community. In this work, we investigate the open-set FAS setting, where the spoof types in testing are unseen. The key challenge is that the unseen PAs are designed to deceive the FAS model and thus appear similar to live faces both at the pixel level and in the feature space. To address this issue, we propose a novel framework for synthesizing unseen PAs and pushing the generated samples toward an open category space. Our approach is motivated by empirical findings that unseen PAs are more likely to be compactly clustered by spoof type and located at the boundary of the live distribution in the spoof-type-aware feature space derived from multi-class optimization. Lastly, we evaluate our method on the SiW-Mv2 cross-type benchmark using both fine-grained and coarse-grained protocols. Compared to the baselines and existing top competitors in close-set or cross-domain settings, our method outperforms them significantly on both protocols.

Existing video recognition algorithms always conduct different training pipelines for inputs with different frame numbers, which requires repetitive training operations and multiplying storage costs. If we evaluate the model using other frames which are not used in training, we observe the performance will drop significantly (see Fig. 1), which is summarized as Temporal Frequency Deviation phenomenon. To fix this issue, we propose a general frame-work, named Frame Flexible Network (FFN), which not only enables the model to be evaluated at different frames to adjust its computation, but also reduces the memory costs of storing multiple models significantly. Concretely, FFN integrates several sets of training sequences, involves Multi-Frequency Alignment (MFAL) to learn temporal frequency invariant representations, and leverages Multi-Frequency Adaptation (MFAD) to further strengthen the representation abilities. Comprehensive empirical validations using various architectures and popular benchmarks solidly demonstrate the effectiveness and generalization of FFN (e.g., 7.08/5.15/2.17% performance gain at Frame 4/8/16 on Something-Something V1 dataset over Uniformer). Code is available at https://github.com/BeSpontaneous/FFN.

Unsupervised domain adaptation (UDA) involves predicting unlabeled data in a target domain by using labeled data from the source domain. However, recent advances in pseudo-labeling (PL) methods have been hampered by noisy pseudo-labels that diminish the local discriminativeness of the target structure. Although neighborhood-based PL can help preserve the local structure, it also risks assigning the whole local neighborhood to the wrong semantic category. To address this issue, we propose a novel framework called neighborhood consistency learning (NCL) that operates at both the semantic and instance levels and features a new consistency objective function. Specifically, our objective function aims to promote semantic consistency in the target neighborhood by computing the correlation matrix between the target samples and their neighborhood aggregation over a batch and matching the correlation matrix to an identity matrix. Importantly, our approach allows the target neighborhood to receive gradients from several potential positive categories instead of just one certain category. Our extensive experiments on UDA benchmarks demonstrate the effectiveness of NCL over other state-of-the-art PL-based methods.

Convolutional Neural Networks have achieved remarkable success in face recognition, in part due to the abundant availability of data. However, the data used for training CNNs is often imbalanced. Prior works largely focus on the long-tailed nature of face datasets in data volume per identity, or focus on single bias variation. In this paper, we show that many bias variations such as ethnicity, head pose, occlusion and blur can jointly affect the accuracy significantly. We propose a sample level weighting approach termed Multi-variation Cosine Margin (MvCoM), to simultaneously consider the multiple variation factors, which orthogonally enhances the face recognition losses to incorporate the importance of training samples. Further, we leverage a learning to learn approach, guided by a held-out meta learning set and use an additive modeling to predict the MvCoM. Extensive experiments on challenging face recognition benchmarks demonstrate the advantages of our method in jointly handling imbalances due to multiple variations.

The topic of generalizing machine learning models learned on a collection of source domains to unknown target domains is challenging. While many domain generalization (DG) methods have achieved promising results, they primarily rely on the source domains at train-time without manipulating the target domains at test-time. Thus, it is still possible that those methods can overfit to source domains and perform poorly on target domains. Driven by the observation that domains are strongly related to styles, we argue that reducing the gap between source and target styles can boost models’ generalizability. To solve the dilemma of having no access to the target domain during training, we introduce Test-time Fourier Style Calibration (TF-Cal) for calibrating the target domain style on the fly during testing. To access styles, we utilize Fourier transformation to decompose features into amplitude (style) features and phase (semantic) features. Furthermore, we present an effective technique to Augment Amplitude Features (AAF) to complement TF-Cal. Extensive experiments on several popular DG benchmarks and a segmentation dataset for medical images demonstrate that our method outperforms state-of-the-art methods.

Despite great progress in supervised image recognition, a large performance drop is usually observed when deploying the model in the wild. Unsupervised domain adaptation (UDA) methods tackle the issue by aligning the source domain and the target domain. However, most existing adversarial based methods attempt to perform the alignment from a holistic view, ignoring the underlying class-level data structure in the target domain. As a result, the representations are distorted by adversarial alignment, leading to a negative transfer. Motivated by this issue, we first claim that this issue can be solved if there exists ‘optimal’ per-sample weights for adversarial alignment, and then devise a meta-learning framework to adaptively learn such adversarial weights. Specifically, we construct a meta-dataset with targetlike distribution as meta knowledge, and use it to guide the learning of the optimal adversarial weights via a meta-learner. By this means, our framework can adaptively adjust the weights of all training samples in adversarial training based on the feedback from meta dataset and thus achieve the categorical-wise domain alignment. We conduct sufficient ablation studies and experiments to show the effectiveness of our approach. Our method is generic to existing domain alignment based methods and could achieve consistently improvements over three UDA classification benchmarks.

As a bridge between videos and natural languages, video-text matching has been a hot multimedia research topic in recent years. Such cross-modal retrieval is usually achieved by learning a common embedding space where videos and text captions are directly comparable. It is still challenging because existing visual representations cannot exploit semantic correlations within videos well, resulting in a mismatch with semantic concepts that are contained in the corresponding text descriptions. In this article, we propose a new Guided Graph Attention Learning (GGAL) model to enhance video embedding learning by capturing important region-level semantic concepts within the spatiotemporal space. Our model builds connections between object regions and performs hierarchical graph reasoning on both frame-level and whole video–level region graphs. During this process, global context is used to guide attention learning on this hierarchical graph topology so that the learned overall video embedding can focus on essential semantic concepts and can be better aligned with text captions. Experiments on commonly used benchmarks validate that GGAL outperforms many recent video-text retrieval methods with a clear margin. As multimedia data in dynamic environments becomes critically important, we also validate GGAL learned video-text representations that can be generalized well to unseen out-of-domain data via cross-dataset evaluations. To further investigate the interpretability of our model, we visualize attention weights learned by GGAL models. We find that GGAL successfully focuses on key semantic concepts in the video and has complementary attention on the context parts based on different ways of building region graphs.

Origami structures can transform their form and function by changing the direction of their folds. This reconfiguration can enable multifunctional robots, but doing so requires a fast, robust, and repeatable actuation method. In this article, we present an origami gripper that uses dynamic transformation to change its kinematic behavior in less than a second. We characterize individual vertices to show that the transformation is predictable and repeatable for different designs and orientations. We then apply it to a multivertex template that is capable of a wide range of shapes and motion patterns, indicating that transformation can be generalized to complex and functional machines. To demonstrate this, we built a transforming origami gripper on a robotic arm to pick up multiple objects. Demonstrations show that the gripper can quickly reconfigure between three different grasping modes and has sufficient stiffness to engage with and lift multiple objects with distinct geometries.

This paper studies Semi-Supervised Domain Adaptation (SSDA), a practical yet under-investigated research topic that aims to learn a model of good performance using unlabeled samples and a few labeled samples in the target domain, with the help of labeled samples from a source domain. Several SSDA methods have been proposed recently, which however fail to fully exploit the value of the few labeled target samples. In this paper, we propose Enhanced Categorical Alignment and Consistency Learning (ECACL), a holistic SSDA framework that incorporates multiple mutually complementary domain alignment techniques. ECACL includes two categorical domain alignment techniques that achieve class-level alignment, a strong data augmentation based technique that enhances the model’s generalizability and a consistency learning based technique that forces the model to be robust with image perturbations. These techniques are applied on one or multiple of the three inputs (labeled source, unlabeled target, and labeled target) and align the domains from different perspectives. ECACL unifies them together and achieves fairly comprehensive domain alignments that are much better than the existing methods: For example, ECACL raises the state-of-the-art accuracy from 68.4 to 81.1 on VisDA2017 and from 45.5 to 53.4 on DomainNet for the 1-shot setting. Our code is available at https://github.com/kailigo/pacl.

Domain generalization aims to learn a model that generalizes to unseen target domains from multiple source domains. Various approaches have been proposed to address this problem by adversarial learning, meta-learning, and data augmentation. However, those methods have no guarantee for target domain generalization. Motivated by an observation that the class-irrelevant information of sample in the form of semantic variation would lead to negative transfer, we propose to linearly disentangle the variation out of sample in feature space and impose a novel class decorrelation regularization on the feature variation. By doing so, the model would focus on the high-level categorical concept for model prediction while ignoring the misleading clue from other variations (including domain changes). As a result, we achieve state-of-the-art performances over all of widely used domain generalization benchmarks, namely PACS, VLCS, Office-Home, and Digits-DG with large margins. Further analysis reveals our method could learn a better domain-invariant representation, and decorrelated feature variation could successfully capture semantic meaning.

While deep neural networks excel in solving visual recognition tasks, they require significant effort to find hyperparameters that make them work optimally. Hyperparameter Optimization (HPO) approaches have automated the process of finding good hyperparameters but they do not adapt to a given task (task-agnostic), making them computationally inefficient. To reduce HPO time, we present HyperSTAR (System for Task Aware Hyperparameter Recommendation), a task-aware method to warm-start HPO for deep neural networks. HyperSTAR ranks and recommends hyperparameters by predicting their performance conditioned on a joint dataset-hyperparameter space. It learns a dataset (task) representation along with the performance predictor directly from raw images in an end-to-end fashion. The recommendations, when integrated with an existing HPO method, make it task-aware and significantly reduce the time to achieve optimal performance. We conduct extensive experiments on 10 publicly available large-scale image classification datasets over two different network architectures, validating that HyperSTAR evaluates 50% less configurations to achieve the best performance compared to existing methods. We further demonstrate that HyperSTAR makes Hyperband (HB) task-aware, achieving the optimal accuracy in just 25% of the budget required by both vanilla HB and Bayesian Optimized HB (BOHB).

This paper presents a robotic gripper design that can perform customizable grasping tasks at the millimeter scale. The design is based on the origami string, a mechanism with a single degree of freedom that can be mechanically programmed to approximate arbitrary paths in space. By using this concept, we create miniature fingers that bend at multiple joints with a single actuator input. The shape and stiffness of these fingers can be varied to fit different grasping tasks by changing the crease pattern of the string. We show that the experimental behavior of these strings follows their analytical models and that they can perform a variety of tasks including pinching, wrapping, and twisting common objects such as pencils, bottle caps, and blueberries.

Self-folding is capable of forming complex three-dimensional structures from planar sheets. However, existing self-folding machines cannot generate large forces or bear high loads. In contrast, traditional robots are expected to operate under large loads but these robots are generally dense, resulting in heavy machines with relatively low strength-to-weight ratios. In this paper, we present a new self-folding technique that is meter-scale and load-bearing. We demonstrate the design and fabrication of both structural beams and pneumatically actuated joints. We model and measure the stiffness and strength of self-folded structural beams. Finally, we integrate these results into a robotic arm that weighs 0.3 kg and can lift up to 1.0 kg. These results indicate that functional, load-bearing, self-folding robots are possible.