Differentiable architecture search (DARTS) is a prevailing direction in automatic machine learning, but it may suffer from performance collapse and generalization issues. Recent efforts mitigate them by integrating regularization into architectural parameters or rule-based operations selection. These efforts primarily emphasize learning the global class-specific features through the image classification task, while overlooking the fine-grained local information during the search process. In this article, we take the first trial to observe that three semantic challenges arise from the classification-based DARTS: 1) inaccurate class-specific features; 2) partial target attention; and 3) blurred semantic regions. To tackle them in one shot, we propose Semantic-DARTS, combining the masked image modeling (MIM) paradigm with the classification task to incorporate local semantic information into the architecture search. Specifically, we design a lightweight reconstruction head that recovers the corrupted image based on the condensed latent feature, which learns both the local semantics and their relationship patch-wisely. Simultaneously, the concurrent classification head strengthens the connection between the global category of the target and the local semantics of their parts. As evidenced by our experiments, the proposed approach achieves state-of-the-art results on CIFAR-10, CIFAR-100, and ImageNet. Furthermore, the searched model is not only able to improve global class-specific features but also to capture fine-grained local representations, improving both the classification performance and the generalization ability.

Continual machine learning in the context of limited computational resources and data availability is critical in the connected digital world. Current intelligent applications predominantly rely on deep learning models requiring labor/computation-intensive training. These models often struggle to adapt effectively to new data while preserving performance on previously learned knowledge. In this paper, we introduce a lightweight method for continual knowledge adaptation that can address these challenges. To prevent disruption of the existing services, we propose a Mixture-of-Experts (MoE) adapter that integrates seamlessly with the existing vision model to encode new data. The weights of the original model are kept fixed during the adaptation process, ensuring the preservation of previously learned knowledge. The MoE technique enables scaling up the parameters of the adapter while maintaining a relatively low computation, making it fit for constrained devices in mobile computation scenarios. Furthermore, to enhance learning efficiency and accelerate convergence with new data, we implement a knowledge fusion mechanism that facilitates interaction between the existing knowledge and the information extracted from new data. The timing of employing the fusion module is further investigated. We find that it is conducive in scenarios where the task's performance requirements are enhanced. The MoE adapter and knowledge fusion module are integrated at each stage with minimal trainable parameters, efficiently optimizing resource usage. Extensive experiments and ablation studies validate the effectiveness of the proposed method. Specifically, the proposed method prevents an accuracy drop of 43.02% on the previous data compared to the continual train method, while achieving an accuracy of 44.81% on the new data, which is even 0.34% higher than fully training a new model.

The demand for continual machine learning in the context of limited computational resources and data availability is critical in the evolving landscape of the connected digital world. Current network applications predominantly rely on deep learning models that require labor/computation-intensive training processes. These models often struggle to effectively adapt to new data while preserving performance on previously acquired knowledge. In this paper, we introduce a lightweight framework for continual knowledge adaptation and learning designed to address these challenges. To prevent disruption of existing services, we propose an attention-based adapter that integrates seamlessly with the existing vision model to encode new incoming data. The weights of the original model are kept fixed during the adaptation process, ensuring the preservation of previously learned knowledge. Furthermore, to enhance learning efficiency and accelerate convergence with new data, we implement a knowledge fusion mechanism that facilitates interaction between existing knowledge and information from new data. Our framework is modular, enabling flexible deployment across distributed devices. The adapter and knowledge fusion module are implemented at each stage with minimal trainable parameters, optimizing resource usage. Extensive experiments and ablation studies validate the effectiveness of the proposed framework.

Neural Architecture Search (NAS) is a powerful tool for automating effective image and video processing DNN designing. The ranking of the accuracy has been advocated to design an efficient performance predictor for NAS. The previous contrastive method solves the ranking problem by comparing pairs of architectures and predicting their relative performance. However, it only focuses on the rankings between the two involved architectures and neglects the overall quality distributions of the search space, which may suffer generalization issues. On the contrary, we propose to let the performance predictor concentrate on the global quality level of specific architecture, and learn the tier embeddings of the whole search space automatically with learnable queries. The proposed method, dubbed as Neural Architecture Ranker with Query-to-Tier technique (NARQ2T), explores the quality tiers of the search space globally and classifies each individual to the tier they belong to. Thus, the predictor gains knowledge of the performance distributions of the search space which helps to generalize its ranking ability to the datasets more easily. Thanks to the encoder-decoder design, our method is able to predict the latency of the searched model without deteriorating the performance prediction. Meanwhile, the global quality distribution facilitates the search phase by directly sampling candidates according to the statistics of quality tiers, which is free of training a search algorithm, e.g., Reinforcement Learning or Evolutionary Algorithm, thus it simplifies the NAS pipeline and saves the computational overheads. The proposed NARQ2T achieves state-of-the-art performance on two widely used datasets for NAS research. Moreover, extensive experiments have validated the efficacy of the designed method.

Neural Architecture Search (NAS) is a powerful tool for automating effective image processing DNN designing. The ranking has been advocated to design an efficient performance predictor for NAS. The previous contrastive method solves the ranking problem by comparing pairs of architectures and predicting their relative performance. However, it only focuses on the rankings between two involved architectures and neglects the overall quality distributions of the search space, which may suffer generalization issues. A predictor, namely Neural Architecture Ranker (NAR) which concentrates on the global quality tier of specific architecture, is proposed to tackle such problems caused by the local perspective. The NAR explores the quality tiers of the search space globally and classifies each individual to the tier they belong to according to its global ranking. Thus, the predictor gains the knowledge of the performance distributions of the search space which helps to generalize its ranking ability to the datasets more easily. Meanwhile, the global quality distribution facilitates the search phase by directly sampling candidates according to the statistics of quality tiers, which is free of training a search algorithm, e.g., Reinforcement Learning (RL) or Evolutionary Algorithm (EA), thus it simplifies the NAS pipeline and saves the computational overheads. The proposed NAR achieves better performance than the state-of-the-art methods on two widely used datasets for NAS research. On the vast search space of NAS-Bench-101, the NAR easily finds the architecture with top 0.01 performance only by sampling. It also generalizes well to different image datasets of NAS-Bench-201, i.e., CIFAR-10, CIFAR-100, and ImageNet-16-120 by identifying the optimal architectures for each of them.