The "pre-training → downstream adaptation" presents both new opportunities and challenges for Continual Learning (CL). Although the recent state-of-the-art in CL is achieved through Parameter-Efficient-Tuning (PET) adaptation paradigm, only prompt has been explored, limiting its application to Transformers only. In this paper, we position prompting as one instantiation of PET, and propose a unified CL framework with general PET, dubbed as Learning-Accumulation-Ensemble (LAE). PET, e.g., using Adapter, LoRA, or Prefix, can adapt a pre-trained model to downstream tasks with fewer parameters and resources. Given a PET method, our LAE framework incorporates it for CL with three novel designs. 1) Learning: the pre-trained model adapts to the new task by tuning an online PET module, along with our adaptation speed calibration to align different PET modules, 2) Accumulation: the task-specific knowledge learned by the online PET module is accumulated into an offline PET module through momentum update, 3) Ensemble: During inference, we respectively construct two experts with online/offline PET modules (which are favored by the novel/historical tasks) for prediction ensemble. We show that LAE is compatible with a battery of PET methods and gains strong CL capability. For example, LAE with Adaptor PET surpasses the prior state-of-the-art by 1.3% and 3.6% in last-incremental accuracy on CIFAR100 and ImageNet-R datasets, respectively. Code is available at https://github.com/gqk/LAE.

Filter pruning, as an effective strategy to obtain efficient compact structures from over-parametric deep neural networks(DNN), has attracted a lot of attention. Previous pruning methods select channels for pruning by developing different criteria, yet little attention has been devoted to whether these criteria can represent correlations between channels. Meanwhile, most existing methods generally ignore the parameters being pruned and only perform additional training on the retained network to reduce accuracy loss. In this paper, we present a novel perspective of re-parametric pruning by Bayesian estimation. First, we estimate the probability distribution of different channels based on Bayesian estimation and indicate the importance of the channels by the discrepancy in the distribution before and after channel pruning. Second, to minimize the variation in distribution after pruning, we re-parameterize the pruned network based on the probability distribution to pursue optimal pruning. We evaluate our approach on popular datasets with some typical network architectures, and comprehensive experimental results validate that this method illustrates better performance compared to the state-of-the-art approaches.

Learning discriminative representation using large-scale face datasets in the wild is crucial for real-world applications, yet it remains challenging. The difficulties lie in many aspects and this work focus on computing resource constraint and long-tailed class distribution. Recently, classification-based representation learning with deep neural networks and well-designed losses have demonstrated good recognition performance. However, the computing and memory cost linearly scales up to the number of identities (classes) in the training set, and the learning process suffers from unbalanced classes. In this work, we propose a dynamic class queue (DCQ) to tackle these two problems. Specifically, for each iteration during training, a subset of classes for recognition are dynamically selected and their class weights are dynamically generated on-the-fly which are stored in a queue. Since only a subset of classes is selected for each iteration, the computing requirement is reduced. By using a single server without model parallel, we empirically verify in large-scale datasets that 10% of classes are sufficient to achieve similar performance as using all classes. Moreover, the class weights are dynamically generated in a few-shot manner and therefore suitable for tail classes with only a few instances. We show clear improvement over a strong baseline in the largest public dataset Megaface Challenge2 (MF2) which has 672K identities and over 88% of them have less than 10 instances. Code is available at https://github.com/bilylee/DCQ

Based on the relaxed search space, differential architecture search (DARTS) is efficient in searching for a high-performance architecture. However, the unbalanced competition among operations that have different trainable parameters causes the model collapse. Besides, the inconsistent structures in the search and retraining stages causes cross-stage evaluation to be unstable. In this paper, we call these issues as an operation gap and a structure gap in DARTS. To shrink these gaps, we propose to induce equalized and consistent optimization in differentiable architecture search (EC-DARTS). EC-DARTS decouples different operations based on their categories to optimize the operation weights so that the operation gap between them is shrinked. Besides, we introduce an induced structural transition to bridge the structure gap between the model structures in the search and retraining stages. Extensive experiments on CIFAR10 and ImageNet demonstrate the effectiveness of our method. Specifically, on CIFAR10, we achieve a test error of 2.39%, while only 0.3 GPU days on NVIDIA TITAN V. On ImageNet, our method achieves a top-1 error of 23.6% under the mobile setting.

Neural network pruning has shown promising performance in reducing computational complexity and facilitate the deployment of deep neural networks on resource-limited edge devices. Most existing pruning methods focus on the indicators of the filter's weight, gradient, or feature map and regard the weak or similar filters as network redundancy. In contrast, the representation of discriminative power is also a fundamental attribute that analog neural networks to have extraordinary performance in various tasks. However, such representation is neglected in existing works. Alternatively, we propose a novel filter pruning strategy via class-wise discriminative power (CDP). Unlike the previous methods, CDP treats the filters that always yield large or small activation values as redundant and reserves the filters that show different magnitudes in activations as they yield high discriminative power. We further propose to obtain such discriminative power by employing the widely-used Term Frequency-Inverse Document Frequency (TF-IDF) on feature representations across classes. Specifically, the output of a filter is considered as a word, and the whole feature map is considered as a document. Then, TF-IDF is used to generate the relevant score between words and all documents. If a filter has low TF-IDF scores is less discriminate and can be pruned. Thus, the filters with high TF-IDF scores are reserved. To our best knowledge, this is the first work that prunes neural networks through class-wise discriminative power and measures such power by introducing TF-IDF in feature representation among different classes. Without any iterative process, CDP achieves better compression trade-offs comparing to the state-of-the-art compression algorithms. For instance, in VGG-16, we achieve a 68.05%-FLOPs reduction, with a 94.86% Top-1 accuracy on CIFAR-10. Specifically, we compress a 90.12%-FLOPs reduction VGG-16, even retains 93.30% Top-1 accuracy on CIFAR-10. The code is available at https://github.com/Tianshuo-Xu/CDP-Towards-Optimal-Filter-Pruning-via-Class-wise-Discriminative-Power.git

Neural architecture search (NAS) advances beyond the state-of-the-art in various computer vision tasks by automating the designs of deep neural networks. In this paper, we aim to address three important questions in NAS: (1) How to measure the correlation between architectures and their performances? (2) How to evaluate the correlation between different architectures? (3) How to learn these correlations with a small number of samples? To this end, we first model these correlations from a Bayesian perspective. Specifically, by introducing a novel Gaussian Process based NAS (GP-NAS) method, the correlations are modeled by the kernel function and mean function. The kernel function is also learnable to enable adaptive modeling for complex correlations in different search spaces. Furthermore, by incorporating a mutual information based sampling method, we can theoretically ensure the high-performance architecture with only a small set of samples. After addressing these problems, training GP-NAS once enables direct performance prediction of any architecture in different scenarios and may obtain efficient networks for different deployment platforms. Extensive experiments on both image classification and face recognition tasks verify the effectiveness of our algorithm.

Highly resolved and accurate air pollution maps are valuable resources for many issues related to air quality including exposure modeling and urban planning. Due to the high equipment costs, there are limited high quality monitoring stations (HQMS) in cities. In order to achieve high resolution air pollution maps, a large number of mobile sensors are required. Besides, mobile sensors require frequent calibrations with the HQMS to maintain data accuracy. Existing work on route design for mobile sensors largely focuses on data reconstruction, which either ignores calibration or views it as an independent problem. To improve the accuracy of data reconstruction, this paper proposes a novel scheme that jointly considers sensor calibration and data reconstruction in route design for mobile sensors. We formulate a novel sensor route planning problem (SRPP) which aims to maximize the mutual information and guarantee the accuracy of measurements through sensor calibration. A heuristic algorithm is proposed to solve the SRPP, which supports calibration between mobile sensors and HQMS in route planning. Simulation results show that, compared with traditional approach, our approach can reduce 83% root mean square error (RMSE) on average.

Real-time accurate air quality data is very important for pollution exposure monitoring and urban planning. However, there are limited high-quality air quality monitoring stations (AQMS) in cities due to their high equipment costs. To provide real-time and accurate data covering large area, this paper proposes a novel scheme that jointly considers online multi-hop calibration and spatio-temporal coverage in route selection for mobile sensors. A novel sensor carrier selection problem (SCSP) is formulated, which aims to maximize the spatio-temporal coverage ratio and guarantee the accuracy of measurements through sensor calibration. An online Bayesian based collaborative calibration (OBCC) scheme is proposed to relax the multi-hop calibration constraint in the SCSP. Based on the OBCC, a multi-hop calibration judgment algorithm (MCJA) is proposed to decide whether the data accuracy of a given set of routes can be guaranteed through collaborative calibration. Furthermore, a heuristic sensor route selection algorithm (SRSA) is then developed to solve the SCSP.

Air pollution seriously affect people's lives, among which PM2.5is especially harmful for humans health. Although many countries have established fixed air quality monitoring stations (AQMS) to monitor air pollution, the costs of constructing and maintaining for AQMS are extremely expensive and the density of AQMS is very low. To acquire fine-grained concentration of PM2.5, this paper have proposed a novel Bayesian based kernel method. Our model leverage heterogeneous data which jointly using images information, camera lens information, GPS information and magnetic sensor information. To study the relationship between PM2.5concentration and images information, we have established a crowdsourcing system and have collected photos for consecutive 16 months. The performance of the proposed method has been evaluated thoroughly by real dataset we have collected. The results show that, compared with three baselines, our proposed algorithm can reduce up to 35% prediction error in average.

Wireless sensor networks (WSN) based on unmanned aerial vehicles (UAV) are ideal platforms for monitoring dynamics over larger service area. On the other hand, aerial sensor networks (ASNs) are often required to be connected with a command center for sending data and receiving control messages in real time. In this paper, we study the problem of path planning for ASNs with connectivity constraints. The primary goal of path planning is driving UAVs to locations where the most informative measurements can be collected. Meanwhile, the worst link's capacity is assured to be greater than a pre-defined requirement. We proposed a solution for the path planning problem and it consists of two modules: network coordinator (NC) and motion controller (MC). The NC manages the topology of relay-assisted wireless communication networks. For the design of MC, we compare two motion strategies: maximum entropy and maximum mutual information. The simulation results show that our proposed solution achieves accurate signal reconstruction while maintaining the connectivity. We conclude that it's important to enable UAV-to-UAV communications for future ASN-based applications.

Mobile Crowd sensing is a new paradigm that encourages ordinary people to collect and share sensing data with their smart devices. However, the uncontrollable data quality is one of the critical problems that is potentially harmful to guarantee the availability and preciseness of mobile crowd sensing based services. In this paper, we propose a quality aware data collection mechanism based on a realistic scenario where participants arrive and report their conditions sequentially one by one. When a participant arrives, the mechanism first forecasts the amount of high quality data he#x002F;she may contribute by employing the expectation of Binomial-Poisson distribution, and then a two-level iterative algorithm is employed to calculate its parametric values. After, our designed mechanism decides to select the participant or not by combining with his#x002F;her requested reward. Extensive simulation results well justify the effectiveness and robustness of our approach, compared with another schemes.

In this paper, the multidimensional output Gaussian process (GP) is applied to model urban environmental data collected by sensor networks. Measurements from sensors at different locations are correlated. Moreover, we observe that the pollution level in urban area is highly coupled with human activities and shows periodic patterns accordingly. Based on these observations, we discuss the design of mean and kernel functions with two approaches: (1) composed kernel and maximum likelihood estimation of hyper-parameters, (2) Wiener-Khinchin theorem based approximation of sample covariances. To validate the models, the accuracy of interpolations given by different approaches are compared. The experimental results show that, for the application of interpolation, the dependent GP with the approximated sample covariances as kernels can provide better performance than the independent GP model with composed kernels.

Location based services are getting increasingly popular in participatory sensing systems. They make use of location information on the mobile devices to support applications that improve personal health, object search, and entertainment. However, GPS positioning consumes a lot of energy, which can drain a mobile device's battery. Although WiFi localization and cell tower localization have been suggested as alternatives, they have lower localization accuracy and limited coverage. In this paper, we suggest a novel solution for multiple mobile devices to perform collaborative localization to reduce energy consumption and provide accurate localization. We divide the mobile devices into two groups, the aggregator group and the collector group. The aggregator group turns on their GPS periodically, while the collector group uses the locations of the aggregators to estimate their own locations. We formulate the aggregator set selection problem and propose two novel algorithms to minimize the energy consumption in collaborative localization. Simulations with real traces showed that our proposed solution can save up to 88% of the energy of the entire network.

The increasingly popular smartphones enable participatory sensing systems to collect location-based sensing data for different tasks. However, GPS positioning is very energy consuming, which could drain a mobile device's battery quickly. High energy consumption may threaten the participants and reduce the sustainability of the participatory sensing systems. In this paper, we propose a collaborative localization strategy with load balancing. Simulations with real traces showed that our proposed solution can save more than 80% of the energy consumption for localization in the entire network with load balancing.