Textile pattern generation (TPG) aims to synthesize fine-grained textile pattern images based on given clothing images. Although previous studies have not explicitly investigated TPG, existing image-to-image models appear to be natural candidates for this task. However, when applied directly, these methods often produce unfaithful results, failing to preserve fine-grained details due to feature confusion between complex textile patterns and the inherent non-rigid texture distortions in clothing images. In this paper, we propose a novel method, SLDDM-TPG, for faithful and high-fidelity TPG. Our method consists of two stages: (1) a latent disentangled network (LDN) that resolves feature confusion in clothing representations and constructs a multi-dimensional, independent clothing feature space; and (2) a semi-supervised latent diffusion model (S-LDM), which receives guidance signals from LDN and generates faithful results through semi-supervised diffusion training, combined with our designed fine-grained alignment strategy. Extensive evaluations show that SLDDM-TPG reduces FID by 4.1 and improves SSIM by up to 0.116 on our CTP-HD dataset, and also demonstrate good generalization on the VITON-HD dataset.

High-quality open-source datasets, which necessitate substantial efforts for curation, has become the primary catalyst for the swift progress of deep learning. Concurrently, protecting these datasets is paramount for the well-being of the data owner. Dataset ownership verification emerges as a crucial method in this domain, but existing approaches are often limited to supervised models and cannot be directly extended to increasingly popular unsupervised pre-trained models. In this work, we propose the first dataset ownership verification method tailored specifically for self-supervised pre-trained models by contrastive learning. Its primary objective is to ascertain whether a suspicious black-box backbone has been pre-trained on a specific unlabeled dataset, aiding dataset owners in upholding their rights. The proposed approach is motivated by our empirical insights that when models are trained with the target dataset, the unary and binary instance relationships within the embedding space exhibit significant variations compared to models trained without the target dataset. We validate the efficacy of this approach across multiple contrastive pre-trained models including SimCLR, BYOL, SimSiam, MOCO v3, and DINO. The results demonstrate that our method rejects the null hypothesis with a $p$-value markedly below $0.05$, surpassing all previous methodologies. Our code is available at https://github.com/xieyc99/DOV4CL.

Huge amounts of data from various sources are substantial to dependable distributed machine learning, especially for trustworthy federated learning (FL). However, existing FL methods are difficult to collect enough data for training the global model more accurately, especially in cross-device scenarios. In this paper, we propose a new federated generative adversarial network empowered by differential privacy and knowledge transfer named Fed-GAN, which can be used to address the problem of data shortage and prevent generator leakage from resource-constrained devices, as well as generate high-quality synthetic data while ensuring strict DP guarantees. Different from other generative model methods, our Fed-GAN framework can achieve efficient and secure generative model training and limited permission for resource-constrained devices to prevent them from leaking or misusing the generator. In addition, we propose a pHash-KT method for our Fed-GAN framework, which selects potentially high-quality data through the knowledge of each client for improving the utility of synthetic data. Our FedGAN framework satisfies ($\frac{2kJ\lambda}{\sigma^2}+\frac{log 1/\delta}{\lambda-1},\delta}$)-DP, and also has high resistance when number of adversaries is 10%-70% of the total number of clients. Extensive experiments demonstrate that our Fed-GAN framework not only generates high-quality synthetic data, but also provides strict DP guarantees, compared with other generative model methods. Our code is publicly available at https://github.com/daxx1/fed-gan

Neural networks (NNs) and decision trees (DTs) are both popular models of machine learning, yet coming with mutually exclusive advantages and limitations. To bring the best of the two worlds, a variety of approaches are proposed to integrate NNs and DTs explicitly or implicitly. In this survey, these approaches are organized in a school which we term neural trees (NTs). This survey aims to present a comprehensive review of NTs and explore in detail how they enhance the model interpretability. Our first contribution is a detailed taxonomy of NTs, which characterizes the seamless integration and co-evolution of NNs and DTs. Subsequently, we analyze NTs in terms of their interpretability and performance and suggest potential solutions to the remaining challenges. Finally, this survey concludes with a discussion about other considerations like conditional computation and promising directions toward this field. A list of papers reviewed in this survey, along with their corresponding codes, is available at: https://github.com/ zju-vipa/awesome-neural-trees.

Concept activation vector (CAV) has attracted broad research interest in explainable AI, by elegantly attributing model predictions to specific concepts. However, the training of CAV often necessitates a large number of high-quality images, which are expensive to curate and thus limited to a predefined set of concepts. To address this issue, we propose Language-Guided CAV (LG-CAV) to harness the abundant concept knowledge within the certain pre-trained vision-language models (e.g., CLIP). This method allows training any CAV without labeled data, by utilizing the corresponding concept descriptions as guidance. To bridge the gap between vision-language model and the target model, we calculate the activation values of concept descriptions on a common pool of images (probe images) with vision-language model and utilize them as language guidance to train the LG-CAV. Furthermore, after training high-quality LG-CAVs related to all the predicted classes in the target model, we propose the activation sample reweighting (ASR), serving as a model correction technique, to improve the performance of the target model in return. Experiments on four datasets across nine architectures demonstrate that LG-CAV achieves significantly superior quality to previous CAV methods given any concept, and our model correction method achieves state-of-the-art performance compared to existing concept-based methods. Our code is available at https://github.com/hqhQAQ/LG-CAV.

To overcome the heterogeneous dynamics and unreliable communication which adversely effect the stable control for connected and automated vehicle platoons, a new intelligent control approach, named virtual order-degradation interconnection and damping assignment (VO-IDA), is proposed in this article. First, the internal stability of vehicle platoons is abstracted into a class of tracking control problems for general chained integral systems. By converting the chained integral system into standard closed-loop port-controlled Hamiltonian form, VO-IDA achieves asymptotic tracking through the integration of backstepping order degradation and virtual stabilization control techniques. This conversion effectively eliminates the dependence on preceding vehicular acceleration as well. Second, under heterogeneous dynamics, explicit stable domains of control parameters are provided to ensure the attenuation of string stability for vehicle platoons via Laplace transform. Furthermore, a linear-proportional relationship between heterogeneous and homogeneous dynamics regarding spacing error ratio is uncovered. Leveraging this relationship, a modified multiobjective genetic algorithm is employed to online explore target locations within stable domains, enabling VO-IDA to conduct stable and precise control under heterogeneous dynamics. Comparative experiments verify the superiority of this approach.

In leader-follower multiagent systems (MASs), seeking an efficient scheme to select a set of agents as leaders is important for realizing the expected cooperative performance. In this article, the problem of minimal leader selection is investigated for impulsive general linear MASs with switching topologies. This study focuses on selecting a set of agents as leaders that receive information from a reference signal directly, while minimizing the number of leaders, subject to consensus tracking performance. First, adopting the average dwell time technique and a time-ratio constraint, an explicit criterion for consensus tracking is derived as prepreparation for leader selection. Second, applying the submodular optimization framework, leader selection metrics are established based on the derived criterion. Third, employing the greedy rule, an efficient leader selection scheme is presented according to the established metrics. The scheme comprises two polynomial-time algorithms that return selected leader sets within a logarithmic bound of the optimum. Finally, the effectiveness of the developed leader selection scheme is verified using an illustrative example.

Most existing online knowledge distillation (OKD) techniques typically require sophisticated modules to produce diverse knowledge for improving students' generalization ability. In this paper, we strive to fully utilize multi-model settings instead of well-designed modules to achieve a distillation effect with excellent generalization performance. Generally, model generalization can be reflected in the flatness of the loss landscape. Since averaging parameters of multiple models can find flatter minima, we are inspired to extend the process to the sampled convex combinations of multi-student models in OKD. Specifically, by linearly weighting students' parameters in each training batch, we construct a Hybrid-Weight Model (HWM) to represent the parameters surrounding involved students. The supervision loss of HWM can estimate the landscape's curvature of the whole region around students to measure the generalization explicitly. Hence we integrate HWM's loss into students' training and propose a novel OKD framework via parameter hybridization (OKDPH) to promote flatter minima and obtain robust solutions. Considering the redundancy of parameters could lead to the collapse of HWM, we further introduce a fusion operation to keep the high similarity of students. Compared to the state-of-the-art (SOTA) OKD methods and SOTA methods of seeking flat minima, our OKDPH achieves higher performance with fewer parameters, benefiting OKD with lightweight and robust characteristics. Our code is publicly available at https://github.com/tianlizhang/OKDPH.

Part-prototype networks (e.g., ProtoPNet, ProtoTree, and ProtoPool) have attracted broad research interest for their intrinsic interpretability and comparable accuracy to non-interpretable counterparts. However, recent works find that the interpretability from prototypes is fragile, due to the semantic gap between the similarities in the feature space and that in the input space. In this work, we strive to address this challenge by making the first attempt to quantitatively and objectively evaluate the interpretability of the part-prototype networks. Specifically, we propose two evaluation metrics, termed as "consistency score" and "stability score", to evaluate the explanation consistency across images and the explanation robustness against perturbations, respectively, both of which are essential for explanations taken into practice. Furthermore, we propose an elaborated part-prototype network with a shallow-deep feature alignment (SDFA) module and a score aggregation (SA) module to improve the interpretability of prototypes. We conduct systematical evaluation experiments and provide substantial discussions to uncover the interpretability of existing part-prototype networks. Experiments on three benchmarks across nine architectures demonstrate that our model achieves significantly superior performance to the state of the art, in both the accuracy and interpretability. Our code is available at https://github.com/hqhQAQ/EvalProtoPNet.

Knowledge amalgamation (KA) is a novel deep model reusing task aiming to transfer knowledge from several well-trained teachers to a multi-talented and compact student. Currently, most of these approaches are tailored for convolutional neural networks (CNNs). However, there is a tendency that Transformers, with a completely different architecture, are starting to challenge the domination of CNNs in many computer vision tasks. Nevertheless, directly applying the previous KA methods to Transformers leads to severe performance degradation. In this work, we explore a more effective KA scheme for Transformer-based object detection models. Specifically, considering the architecture characteristics of Transformers, we propose to dissolve the KA into two aspects: sequence-level amalgamation (SA) and task-level amalgamation (TA). In particular, a hint is generated within the sequence-level amalgamation by concatenating teacher sequences instead of redundantly aggregating them to a fixed-size one as previous KA approaches. Besides, the student learns heterogeneous detection tasks through soft targets with efficiency in the task-level amalgamation. Extensive experiments on PASCAL VOC and COCO have unfolded that the sequence-level amalgamation significantly boosts the performance of students, while the previous methods impair the students. Moreover, the Transformer-based students excel in learning amalgamated knowledge, as they have mastered heterogeneous detection tasks rapidly and achieved superior or at least comparable performance to those of the teachers in their specializations.

Continual learning is a longstanding research topic due to its crucial role in tackling continually arriving tasks. Up to now, the study of continual learning in computer vision is mainly restricted to convolutional neural networks (CNNs). However, recently there is a tendency that the newly emerging vision transformers (ViTs) are gradually dominating the field of computer vision, which leaves CNN-based continual learning lagging behind as they can suffer from severe performance degradation if straightforwardly applied to ViTs. In this paper, we study ViT-backed continual learning to strive for higher performance riding on recent advances of ViTs. Inspired by mask-based continual learning methods in CNNs, where a mask is learned per task to adapt the pre-trained ViT to the new task, we propose MEta-ATtention (MEAT), i.e., attention to self-attention, to adapt a pre-trained ViT to new tasks without sacrificing performance on already learned tasks. Unlike prior mask-based methods like Piggyback, where all parameters are associated with corresponding masks, MEAT leverages the characteristics of ViTs and only masks a portion of its parameters. It renders MEAT more efficient and effective with less overhead and higher accuracy. Extensive experiments demonstrate that MEAT exhibits significant superiority to its state-of-the-art CNN counterparts, with 4.0 ∼ 6.0% absolute boosts in accuracy. Our code has been released at https://github.com/zju-vipa/MEAT-TIL.

Recently, vision Transformers (ViTs) are developing rapidly and starting to challenge the domination of con-volutional neural networks (CNNs) in the realm of computer vision (CV). With the general-purpose Transformer architecture replacing the hard-coded inductive biases of convolution, ViTs have surpassed CNNs, especially in data-sufficient circumstances. However, ViTs are prone to over-fit on small datasets and thus rely on large-scale pre-training, which expends enormous time. In this paper, we strive to liberate ViTs from pre-training by introducing CNNs' in- ductive biases back to ViTs while preserving their network architectures for higher upper bound and setting up more suitable optimization objectives. To begin with, an agent CNN is designed based on the given ViT with inductive bi-ases. Then a bootstrapping training algorithm is proposed to jointly optimize the agent and ViT with weight sharing, during which the ViT learns inductive biases from the intermediate features of the agent. Extensive experiments on CIFAR-10/100 and ImageNet-1k with limited training data have shown encouraging results that the inductive biases help ViTs converge significantly faster and outperform conventional CNNs with even fewer parameters. Our code is publicly available at https://github.com/zhfeing/Bootstrapping-ViTs-pytorch.

Knowledge distillation pursues a diminutive yet well-behaved student network by harnessing the knowledge learned by a cumbersome teacher model. Prior methods achieve this by making the student imitate shallow behaviors, such as soft targets, features, or attention, of the teacher. In this paper, we argue that what really matters for distillation is the intrinsic problem-solving process captured by the teacher. By dissecting the decision process in a layer-wise manner, we found that the decision-making procedure in the teacher model is conducted in a coarse-to-fine manner, where coarse-grained discrimination (e.g., animal vs vehicle) is attained in early layers, and fine-grained dis-crimination (e.g., dog vs cat, car vs truck) in latter layers. Motivated by this observation, we propose a new distillation method, dubbed as Tree-like Decision Distillation (TDD), to endow the student with the same problem-solving mechanism as that of the teacher. Extensive experiments demonstrated that TDD yields competitive performance compared to state of the arts. More importantly, it enjoys better interpretability due to its interpretable decision distillation instead of dark knowledge distillation.

Knowledge distillation (KD) has recently emerged as an efficacious scheme for learning compact deep neural networks (DNNs). Despite the promising results achieved, the rationale that interprets the behavior of KD has yet remained largely understudied. In this paper, we introduce a novel task-oriented attention model, termed as KDExplainer, to shed light on the working mechanism underlying the vanilla KD. At the heart of KDExplainer is a Hierarchical Mixture of Experts (HME), in which a multi-class classification is reformulated as a multi-task binary one. Through distilling knowledge from a free-form pre-trained DNN to KDExplainer, we observe that KD implicitly modulates the knowledge conflicts between different subtasks, and in reality has much more to offer than label smoothing. Based on such findings, we further introduce a portable tool, dubbed as virtual attention module (VAM), that can be seamlessly integrated with various DNNs to enhance their performance under KD. Experimental results demonstrate that with a negligible additional cost, student models equipped with VAM consistently outperform their non-VAM counterparts across different benchmarks. Furthermore, when combined with other KD methods, VAM remains competent in promoting results, even though it is only motivated by vanilla KD. The code is available at https:// github.com/zju-vipa/KDExplainer.

A massive number of well-trained deep networks have been released by developers online. These networks may focus on different tasks and in many cases are optimized for different datasets. In this paper, we study how to exploit such heterogeneous pre-trained networks, known as teachers, so as to train a customized student network that tackles a set of selective tasks defined by the user. We assume no human annotations are available, and each teacher may be either single- or multi-task. To this end, we introduce a dual-step strategy that first extracts the task-specific knowledge from the heterogeneous teachers sharing the same sub-task, and then amalgamates the extracted knowledge to build the student network. To facilitate the training, we employ a selective learning scheme where, for each unlabelled sample, the student learns adaptively from only the teacher with the least prediction ambiguity. We evaluate the proposed approach on several datasets and the experimental results demonstrate that the student, learned by such adaptive knowledge amalgamation, achieves performances even better than those of the teachers.

This paper is concerned with the stabilization problem of type-2 fuzzy systems with network-induced packet losses. By regarding the packet lost process as an unstable mode of a switched system, the stability of the system is then guaranteed with the aid of the mode-dependent average dwell time approach in the sense of the slow and fast switching. The discrete-time multiple discontinuous Lyapunov function is also utilized for the analysis. Two sufficient conditions regarding the stability and the stabilization of the system are proposed. The state-feedback matrices can be then calculated from the conditions to ensure the criterion that the packet-loss rate is no larger than a specific constant. Two practical examples are given to illustrate the feasibility and effectiveness of the proposed method.

The model approximation problem is studied in this paper for switched genetic regulatory networks (GRNs) with time-varying delays. We focus on constructing a reduced-order model to approximate the high-order GRNs considered under the switching signal subject to certain constraints, such that the approximation error system between the original and reduced-order systems is exponentially stable with a disturbance attenuation performance. The stability conditions and the disturbance attenuation performance are established by utilizing two integral inequality bounding techniques and the average dwell-time method for the approximation error system. Then, the solvability conditions for the reduced-order models for the GRNs are also established using the projection method. Furthermore, the model approximation problem can be transferred into a sequential minimization problem that is subject to linear matrix inequality constraints by using the cone complementarity algorithm. Finally, several examples are provided to illustrate the effectiveness and the advantages of the proposed methods.

In this work, the control problem for type-2 T-S fuzzy system with packet dropouts is investigated by modeling the system as a switched system with an unstable subsystem. The mode-dependent average dwell time approach in both slow and fast switching sense is utilized for the analysis and synthesis. A sufficient condition is given by ensuring the packet loss rate no bigger than the specific fast switching mode-dependent average dwell time (MDADT) and the corresponding feedback matrices are obtained. Several simulation results illustrate the feasibility and effectiveness of the proposed method and the priority of the type-2 fuzzy system on describing some nonlinear systems.