As a challenging problem, few-shot class-incremental learning (FSCIL) continually learns a sequence of tasks, confronting the dilemma between slow forgetting of old knowledge and fast adaptation to new knowledge. In this paper, we concentrate on this "slow versus fast" (SvF) dilemma to determine which knowledge components to be updated in a slow fashion or a fast fashion, and thereby balance old-knowledge preservation and new-knowledge adaptation. We propose a multi-grained SvF learning strategy to cope with the SvF dilemma from two different grains: intra-space (within the same feature space) and inter-space (between two different feature spaces). The proposed strategy designs a novel frequency-aware regularization to boost the intra-space SvF capability, and meanwhile develops a new feature space composition operation to enhance the inter-space SvF learning performance. With the multi-grained SvF learning strategy, our method outperforms the state-of-the-art approaches by a large margin.

Different from universal object detection, referring expression comprehension (REC) aims to locate specific objects referred to by natural language expressions. The expression provides high-level concepts of relevant visual and contextual patterns, which vary significantly with different expressions and account for only a few of those encoded in the REC model. This leads us to a question: do we really need the entire network with a fixed structure for various referring expressions? Ideally, given an expression, only expression-relevant components of the REC model are required. These components should be small in number as each expression only contains very few visual and contextual clues. This paper explores the adaptation between expressions and REC models for dynamic inference. Concretely, we propose a neat yet efficient framework named Language Adaptive Dynamic Subnets (LADS), which can extract language-adaptive subnets from the REC model conditioned on the referring expressions. By using the compact subnet, the inference can be more economical and efficient. Extensive experiments on RefCOCO, RefCOCO+, RefCOCOg, and Referit show that the proposed method achieves faster inference speed and higher accuracy against state-of-the-art approaches.

As a key problem of auto-vehicle applications, the goal of Anomaly Obstacle Segmentation (AOS) is to detect some strange and unexpected obstacles (possibly are unseen previously) on the drivable area, thereby equipping the semantic perceptual model to be tolerant of unknown things. Due to its practicality, recently AOS is drawing attentions and a long line of works are proposed to tackle the obstacles with almost infinite diversity. However, these methods usually focus less on the priors of driving scenarios and involve image re-generation or the retraining of perceptual model, which lead to large computational quantity or the degradation of perceptual performance. In this paper, we propose to pay more attention to the characteristics of driving scenarios, lowering the difficulty of this tricky task. A training-free retrieval based method is thereby proposed to distinguish road obstacles from the surrounding road texture by computing the cosine similarity based on their appearance features, and significantly outperforms methods of the same category by around 20 percentage points. Besides, we find that there is a deep relation between our method and self-attention mechanism, and as a result a novel Transformer evolves from our retrieval based method, further boosting the performance.

Model performance can be further improved with the extra guidance apart from the one-hot ground truth. To achieve it, recently proposed recollection-based methods utilize the valuable information contained in the past training history and derive a "recollection" from it to provide data-driven prior to guide the training. In this article, we focus on two fundamental aspects of this method, i.e., recollection construction and recollection utilization. Specifically, to meet the various demands of models with different capacities and at different training periods, we propose to construct a set of recollections with diverse distributions from the same training history. After that, all the recollections collaborate together to provide guidance, which is adaptive to different model capacities, as well as different training periods, according to our similarity-based elastic knowledge distillation (KD) algorithm. Without any external prior to guide the training, our method achieves a significant performance gain and outperforms the methods of the same category, even as well as KD with well-trained teacher. Extensive experiments and further analysis are conducted to demonstrate the effectiveness of our method.

With the memory-resource-limited constraints, class-incremental learning (CIL) usually suffers from the "catastrophic forgetting" problem when updating the joint classification model on the arrival of newly added classes. To cope with the forgetting problem, many CIL methods transfer the knowledge of old classes by preserving some exemplar samples into the size-constrained memory buffer. To utilize the memory buffer more efficiently, we propose to keep more auxiliary low-fidelity exemplar samples, rather than the original real-high-fidelity exemplar samples. Such a memory-efficient exemplar preserving scheme makes the old-class knowledge transfer more effective. However, the low-fidelity exemplar samples are often distributed in a different domain away from that of the original exemplar samples, that is, a domain shift. To alleviate this problem, we propose a duplet learning scheme that seeks to construct domain-compatible feature extractors and classifiers, which greatly narrows down the above domain gap. As a result, these low-fidelity auxiliary exemplar samples have the ability to moderately replace the original exemplar samples with a lower memory cost. In addition, we present a robust classifier adaptation scheme, which further refines the biased classifier (learned with the samples containing distillation label knowledge about old classes) with the help of the samples of pure true class labels. Experimental results demonstrate the effectiveness of this work against the state-of-the-art approaches. We will release the code, baselines, and training statistics for all models to facilitate future research.

As an important and challenging problem, multidomain learning (MDL) typically seeks a set of effective lightweight domain-specific adapter modules plugged into a common domain-agnostic network. Usually, existing ways of adapter plugging and structure design are handcrafted and fixed for all domains before model learning, resulting in learning inflexibility and computational intensiveness. With this motivation, we propose to learn a data-driven adapter plugging strategy with neural architecture search (NAS), which automatically determines where to plug for those adapter modules. Furthermore, we propose an NAS-adapter module for adapter structure design in an NAS-driven learning scheme, which automatically discovers effective adapter module structures for different domains. Experimental results demonstrate the effectiveness of our MDL model against existing approaches under the conditions of comparable performance.

Accurate detection of objects in perception system is a basic task for autonomous vehicles to operate reliably. The current mainstream paradigms for perception and prediction are able to decompose into three sequential subtasks: detection, tracking and prediction. Much of the previous works focus on one of these subtasks and internal relationship between the various subtasks are therefore ignored. In this work we propose an end-to-end model that is able to jointly carry out 3D detection and motion prediction in the context of autonomous vehicles, which can benefit from joint optimization of two tasks. Considering computational consumption and memory cost, our approach utilizes 2D convolutions across space over a bird's eye view (BEV) representation of the 3D world, not 3D convolutions which require more inference time for a single point cloud. In this paper, the key model is the sequential multi-frame point clouds channels based on BEV, which can help perform object detection and learn temporal information at the same time. Experiments on nuScenes dataset show that the proposed network achieves state-of-the-art results, and the trade-off between accuracy and efficiency of two tasks are obtained. Additionally, we validate the effectiveness of our models through the ablation studies. Specifically, by sharing network, parameters and detection results we can perform two tasks within 32 FPS.

With the rapid development of social media, tremendous videos with new classes are generated daily, which raise an urgent demand for video classification methods that can continuously update new classes while maintaining the knowledge of old videos with limited storage and computing resources. In this paper, we summarize this task as Class-Incremental Video Classification (CIVC) and propose a novel framework to address it. As a subarea of incremental learning tasks, the challenge of catastrophic forgetting is unavoidable in CIVC. To better alleviate it, we utilize some characteristics of videos. First, we decompose the spatio-temporal knowledge before distillation rather than treating it as a whole in the knowledge transfer process; trajectory is also used to refine the decomposition. Second, we propose a dual granularity exemplar selection method to select and store representative video instances of old classes and key-frames inside videos under a tight storage budget. We benchmark our method and previous SOTA class-incremental learning methods on Something-Something V2 and Kinetics datasets, and our method outperforms previous methods significantly.

As an important and challenging problem, gait recognition has gained considerable attention. It suffers from confounding conditions, that is, it is sensitive to camera views, dressing types and so on. Interestingly, it is observed that, under different conditions, local body parts contribute differently to recognition performance. In this paper, we propose a condition-aware comparison scheme to measure gait pairs' similarity via a novel module named Instructor. Also, we present a geometry-guided data augmentation approach (Dresser) to enrich dressing conditions. Furthermore, to enhance the gait representation, we propose to model temporal local information from coarse to fine. Our model is evaluated on two popular benchmarks, CASIA-B and OULP. Results show that our method outperforms current state-of-the-art methods, especially in the cross-condition scenario.

Different from many other attributes, facial expression can change in a continuous way, and therefore, a slight semantic change of input should also lead to the output fluctuation limited in a small scale. This consistency is important. However, current Facial Expression Recognition (FER) datasets may have the extreme imbalance problem, as well as the lack of data and the excessive amounts of noise, hindering this consistency and leading to a performance decreasing when testing. In this paper, we not only consider the prediction accuracy on sample points, but also take the neighborhood smoothness of them into consideration, focusing on the stability of the output with respect to slight semantic perturbations of the input. A novel method is proposed to formulate semantic perturbation and select unreliable samples during training, reducing the bad effect of them. Experiments show the effectiveness of the proposed method and state-of-the-art results are reported, getting closer to an upper limit than the state-of-the-art methods by a factor of 30% in AffectNet, the largest in-the-wild FER database by now.