Currently, pretrained models are rapidly scaling in size, which substantially increases the cost of fine-tuning them for downstream tasks. To address this challenge, parameter-efficient fine-tuning (PEFT) methods have been developed to optimize a minimal set of parameters for adaptation. While current PEFT approaches predominantly employ an "additive'' strategy, introducing learnable modules into inputs or architectures, neglect the inherent knowledge embedded within pretrained models, which may be redundant or even conflict with downstream tasks. This limitation leads to increased inference latency and suboptimal transfer performance, particularly in scenarios with significant domain gaps. In this paper, we propose a Subtractive Fine-tuning Paradigm(SFP), which converts multiple redundant operations within the original module into a linear transformation to enhance inference speed and model performance. Specifically, we introduce a compact filter block to replace specific module with interference and redundancy in the original structure to reduce model conflicts. By using a pseudo inverse matrix to construct filter block, ensuring that it can inherit the knowledge of the replacement module, and then freezing the rest of the model, only fine-tuning the filter block is performed to eliminate interference and redundant knowledge, thereby enhancing the model’s adaptability to downstream tasks. Experimental results demonstrate that our SFP outperforms existing PEFT methods in accuracy while decreasing the overall model parameters by 12%. Compared to full fine-tuning, the accuracy has increased by 8.47%(74.04% vs. 65.57%, VTAB).

Federated learning (FL) enables privacy-preserving collaboration among distributed clients, but practical deployments often face heterogeneous models and non-IID data, leading to degraded communication and personalization. In addition, real-world FL systems frequently encounter newly joined clients that require rapid adaptation and abnormal clients that may upload corrupted updates, further exacerbating instability and hindering global convergence. To address these challenges in image classification, we propose HFedDGHN, a Heterogeneous Federated Dynamic Graph HyperNetwork that jointly models inter-client relations and personalized parameter generation. Specifically, a graph structure learner adaptively captures client correlations to construct a dynamic collaboration graph, while a graph-convolutional hypernetwork generates model parameters for heterogeneous architectures, enabling implicit knowledge transfer without sharing local data or weights. Moreover, the framework naturally supports meta-learning-based generalization, allowing efficient adaptation to newly joined clients. Furthermore, the dynamic graph enhances robustness by isolating abnormal clients, as they tend to be excluded from most neighborhoods during adaptive graph construction. Extensive experiments across multiple benchmarks demonstrate that HFedDGHN achieves superior accuracy compared to state-of-the-art personalized and heterogeneous FL methods, while naturally improving robustness and scalability in real-world deployments.

Multi-granularity classification assigns labels of different granularities to each instance. Fine-grained label classification relies on costly expert annotations, making it essential to train high-performance models under weak supervision conditions. Inspired by how humans progressively classify from generalized concepts to fine-grained details while considering independence and correlation in topological structures, a weakly supervised multi-granularity learning guided by semantic topological knowledge is proposed. This approach requires only minimal fine-grained annotations to achieve high-performance classification through progressive learning. Specifically, the model first extracts local features of different granularity from global features based on edge attention. Then, the coarse-grained features guide the feature learning of other granularities and fuse the global and local features. Finally, progressive classification is realized based on Markov logic using multi-granularity topology. Experiments demonstrate that the proposed method performs well under weakly supervised conditions, especially when the label missing rate is as high as 90%, and the performance is improved by 23.59% over the comparison method.

Federated unlearning enables federated learning systems to forget the learned knowledge, aims to guarantee the "right to be forgotten" of clients. However, federated continual learning as a new federated learning paradigm brings new challenges to unlearning. Particularly, unlearned knowledge can be mixed with relevant knowledge from other clients or local learned tasks, removing unlearned knowledge is prone to cause excessive forgetting of relevant knowledge and consequently harming the model’s performance on the remain set. Therefore, we propose a novel unlearning method named relevant knowledge-aware unlearning (RKU), which introduces relevant knowledge-aware interference to address excessive forgetting of relevant knowledge during the unlearning process, and introduces selective knowledge retrieval to address the catastrophic forgetting of remain knowledge while avoiding undermining unlearning efficiency. We validate the effectiveness of RKU by comparing it with state-of-the-art federated unlearning methods on CIFAR100 and TinyImageNet datasets. The code is available at https://github.com/zhanghad/RKU.

Federated continual learning (FCL) as an emerging learning paradigm empowers federated learning (FL) to continuously learn new tasks, allowing it to adapt to dynamic scenarios. Despite many FCL studies have emerged, most of them focus on the catastrophic forgetting and neglect the challenges brought by heterogeneity, which hinders FCL methods from being applied to real-world scenarios. FCL presents new challenges in both data and model heterogeneity compared to FL, including data heterogeneity in time and space dimensions and model heterogeneity in output and weight spaces. We propose a novel approach called global and local rebalance (GLRB) to address the challenges of model and data heterogeneity. We propose a unified feature space for knowledge exchange in the presence of model heterogeneity, and two communication strategies are proposed to accommodate different communication conditions We demonstrate the effectiveness of GLRB through comparison and ablation experiments on multiple datasets.

Federated Learning is a distributed paradigm that facilitates collaborative training of deep models among multiple parties without exchanging raw data. However, the common non-independent and identically distributed (Non-IID) data distribution among clients introduces discrepancies between local training objectives and the global goal. This misalignment results in a slow convergence of global model and a decrease in generalization performance. We propose a method to enhance consistent federated learning objectives through uniform and consistent feature distributions (FedUF). FedUF effectively captures the feature variation space rich in semantic information and integrates implicit semantic data augmentation and logits adjustment to establish a uniform and consistent global feature distribution. Through lightweight yet innovative adjustments to the client-side objective functions, we formulate a globally consistent objective function. The mutual promotion between globally consistent feature distribution and the objective function significantly alleviates the impact of Non-IID data, greatly enhancing the overall performance of Federated Learning. Moreover, extensive experiments conducted on Cifar10 and Cifar100 datasets convincingly validate the effectiveness of the proposed FedUF.

Federated active learning has been proposed as a potential solution to limited labeling budget in federated learning. The global model contains balanced knowledge across clients, while the local model provides a better fit to local knowledge. As the number of locally labeled samples increases, the local model can be iterated locally, resulting in a richer local knowledge. Therefore, using the predictive consistency of the global model and the iterative local models to filter pseudo-labels can improve the correctness of pseudo-labels. Based on this, we propose unnecessary budget reduction(UBR). UBR provides the oracle with inconsistent candidate samples to request true labels, while consistent candidate samples are given pseudo-labels. In order to maintain the model performance, candidate samples with pseudo labels are added to the model training. A progressive training strategy is employed to avoid pseudo-labeling noise affecting model training, in which pseudo-labeled samples are added to model training only when the pseudo-labeling correctness rate is sufficiently high. We conduct experiments combining UBR with several federated active learning methods, which demonstrates UBR effectively reduces the unnecessary query budget while maintaining model performance.

Domain adaptation mitigates the decline in performance that occurs when models are utilized in a target domain. Models designed for a limited range of categories struggle to handle real-world scenarios where unknown classes, absent from the original domain, exist. Furthermore, it is probable that multiple source domains are annotated asynchronously by distinct agencies, each with its own data distributions. The practical challenges of multi-source open-set domain adaptation (MSOSDA) have not been thoroughly investigated, despite their relevance in real-world scenarios. The main difficulty in MSOSDA lies in developing a shared discriminative feature space across all domains, while effectively separating source classes from target-specific ones. In this study, we propose a method for MSOSDA using a self-supervised vision Transformer (ViT) combined with nearest neighbor classification. Our key insight is to leverage the powerful nearest neighbor classification property of self-supervised ViT, along with supervised contrastive learning. To explicitly align the domains and accurately identify unknown classes in the target domain, we employ straightforward strategies and an adaptive data-driven threshold. Our approach has been extensively evaluated on five multi-source domain adaptation benchmarks, showcasing its effectiveness. Among these benchmarks, two are fine-grained, and it is worth noting that one of them has been introduced for the first time in this paper. Through these experiments, we provide compelling evidence of the performance and efficacy of our proposed approach.

Recently, class incremental semantic segmentation (CISS) towards the practical open-world setting has attracted increasing research interest, which is mainly challenged by the well-known issue of catastrophic forgetting. Particularly, knowledge distillation (KD) techniques have been widely studied to alleviate catastrophic forgetting. Despite the promising performance, existing KD-based methods generally use the same distillation schemes for different intermediate layers to transfer old knowledge, while employing manually tuned and fixed trade-off weights to control the effect of KD. These KD-based methods take no consideration of feature characteristics from different intermediate layers, limiting the effectiveness of KD for CISS. In this paper, we propose a layer-specific knowledge distillation (LSKD) method to assign appropriate knowledge schemes and weights for various intermediate layers by considering feature characteristics, aiming to further explore the potential of KD in improving the performance of CISS. Specifically, we present a mask-guided distillation (MD) to alleviate the background shift on semantic features, which performs distillation by masking the features affected by the background. Furthermore, a mask-guided context distillation (MCD) is presented to explore global context information lying in high-level semantic features. Based on them, our LSKD assigns different distillation schemes according to feature characteristics. To adjust the effect of layer-specific distillation adaptively, LSKD introduces a regularized gradient equilibrium method to learn dynamic trade-off weights. Additionally, our LSKD makes an attempt to simultaneously learn distillation schemes and trade-off weights of different layers by developing a bi-level optimization method. Extensive experiments on widely used Pascal VOC 12 and ADE20K show our LSKD clearly outperforms its counterparts while achieving state-of-the-art results.

Open set domain adaptation (OSDA) methods have been proposed to leverage the difference between the source and target domains, as well as to recognize the known and unknown classes in the target domain. Such methods typically require the entire source and target data simultaneously to train the target model. However, in real scenarios, data are distributed and stored in various clients. They cannot be exchanged among clients because of privacy protection. Federated learning (FL) is a decentralized approach for training an effective global model with the training data distributed among the clients. Despite its potential in addressing the privacy concerns of data sharing, FL methods for OSDA that can handle unknown classes is not yet available. To tackle this problem, we have developed a novel federated OSDA (FOSDA) algorithm. More specifically, FOSDA adopts an uncertainty-aware mechanism to generate a global model from all client models. It reduces the uncertainty of the federated aggregation by focusing on the contribution of source clients with high uncertainty while retaining those with high consistency. Moreover, a federated class-based weighted strategy is also implemented in FOSDA to maintain the category information of the source clients. We have conducted comprehensive experiments on three benchmark datasets to evaluate the performance of the proposed method, and the results demonstrate the effectiveness of FOSDA.

Federated learning can coordinate multiple users to participate in data training while ensuring data privacy. The collaboration of multiple agents allows for a natural connection between federated learning and collective intelligence. When there are large differences in data distribution among clients, it is crucial for federated learning to design a reliable client selection strategy and an interpretable client communication framework to better utilize group knowledge. Herein, a reliable personalized federated learning approach, termed RIPFL, is proposed and fully interpreted from the perspective of social learning. RIPFL reliably selects and divides the clients involved in training such that each client can use different amounts of social information and more effectively communicate with other clients. Simultaneously, the method effectively integrates personal information with the social information generated by the global model from the perspective of Bayesian decision rules and evidence theory, enabling individuals to grow better with the help of collective wisdom. An interpretable federated learning mind is well scalable, and the experimental results indicate that the proposed method has superior robustness and accuracy than other state-of-the-art federated learning algorithms.

Open-set domain adaptation (OSDA) aims to align shared classes between the source and the target domain and recognize the private classes of the target domain as unknown. Although unknown classes represent semantic novelty, current OSDA benchmarks lack clear definitions of semantic categories. We propose to use fine-grained visual categorization (FGVC) datasets for the issue because of their specific descriptions of semantic classes. This introduces the new setting named fine-grained OSDA. The entanglement among FGVC, unknown class recognition, and domain adaptation makes fine-grained OSDA a challenging problem. In this paper, we propose a multi-granularity discriminative adversarial network. It utilizes multi-grained labels of the source domain and curriculum learning to improve FGVC performance, exploits discriminative information to recognize unknown classes, and adapts domains through a conditional domain discriminator. Extensive experiments demonstrate our approach outperforms the state-of-the-art methods.

Autonomous driving scene recognition is a challenging and promising task that aims to locate and classify the temporal regions where driving scenes may occur. Most of the existing methods use the methods of temporal action proposal generation: the temporal scene of autonomous driving is seen as an action for recognition. However, due to the complexity and similarity of scene conditions, the application of temporal action recognition methods to autonomous driving scenes is not as effective. To solve these difficulties, based on the fact that autonomous driving scenes are subject to certain rules, we propose a rule-assisted mechanism to help classification, which is based on the fact that the temporal order of autonomous driving scenes is governed by traffic rules and realistic situations, so that the classification results that violate the rules are cancelled out, thus improving the effectiveness of classification. In the rule-constrained approach, we propose two rule formulation methods, one is to manually formulate a perfect rule to which all the temporal order scenes must conform, and the other is to generate rules in the training dataset using the algorithm we propose. We conducted experiments on two autonomous driving datasets provided by CAC data centers, and in combination with existing action classifiers, our approach shows significant performance improvements and is scalable.

Deep neural networks usually depend on substantial labeled data and suffer from poor generalization to new domains. Domain adaptation can be used to resolve these issues, using a classifier trained with a label-rich source and transferred to a label-scarce target domain. Traditional domain adaptation adopts the close-set assumption that both domains share the same classes. However, real-world applications operate in an open-set scenario where target domains have private categories. This aspect is considered by open-set domain adaptation (OSDA). Nevertheless, current OSDA benchmarks lack clear definitions of semantic classes that are at the core of the open-set concept. In this study, we propose fine-grained visual categorization (FGVC) datasets containing specific descriptions of semantic classes as a solution, introducing the new setting named fine-grained OSDA. Owing to the entanglement among FGVC, unknown class recognition, and domain adaptation, fine-grained OSDA is a challenging task. For this reason, we designed a weighted discriminative adversarial network with dual classifiers (WDAN). It utilizes a selective transformer encoder with overlapping patches and supervised contrastive learning to extract features suitable for FGVC, adversarial training with domain-specific discriminative information to recognize target-private classes, and a weighted conditional domain discriminator to learn domain-invariant features for domain adaptation. Extensive experiments on five benchmarks, including one newly built, demonstrated that WDAN outperforms state-of-the-art methods. This work fills the existing gap in benchmarks for fine-grained OSDA, promoting future developments of real-world applications.

Unsupervised multi-source domain adaptation aims to obtain a model working well on the unlabeled target domain by reducing the domain gap between the labeled source domains and the unlabeled target domain. Considering the data privacy and storage cost, data from multiple source domains and target domain are isolated and decentralized. This data decentralization scenario brings the difficulty of domain alignment for reducing the domain gap between the decentralized source domains and target domain, respectively. For conducting domain alignment under the data decentralization scenario, we propose Multi-source Collaborative Contrastive learning for decentralized Domain Adaptation (MCC-DA). The models from other domains are used as the bridge to reduce the domain gap. On the source domains and target domain, we penalize the inconsistency of data features extracted from the source domain models and target domain model by contrastive alignment. With the collaboration of source domain models and target domain model, the domain gap between decentralized source domains and target domain is reduced without accessing the data from other domains. The experiment results on multiple benchmarks indicate that our method can reduce the domain gap effectively and outperform the state-of-the-art methods significantly.

Filter pruning is one of the most popular approaches for compressing convolutional neural networks (CNNs). The most critical task in pruning is to evaluate the importance of each convolutional filter, such that the less important filters can be removed while the overall model performance is minimally affected. In each layer, some filters may be linearly dependent on each other, which means that they have replaceable information. Redundant information can be removed without significantly affecting information richness and model performance. In this paper, we propose a novel low-rank guided pruning scheme to obtain skeleton neural networks by alternatively training and pruning CNNs. In each step, training is performed with nuclear-norm regularization to low-rank the filters in each layer, followed by filter pruning to maintain the information richness via the maximally linearly independent subsystem. A novel “smaller-norm-and-linearly-dependent-less-important” pruning criterion is proposed to compress the model. The training and pruning processes can be repeated until the model is fully trained. To investigate the performance, we applied the proposed joint training and pruning scheme to train the CNNs for image classification. We considered three benchmark datasets: MNIST, CIFAR-10 and ILSVRC-2012. The proposed method successfully achieved a higher pruning rate and better classification performance compared to state-of-the-art compression methods.

As a key task in autonomous driving, 3D object detection based on LiDAR-camera fusion is expected to achieve more robust results by the complementarity of the two sensors. However, LiDAR-camera fusion is non-trivial. An existing problem for this type of detector is that the scale and receptive field of LiDAR point features and image features are not matched, leading to information deficiency or redundancy in fusion. This paper proposes a Point-based Pyramid Attention Fusion (PPAF) module for LiDAR-camera fusion to solve the problem. The PPAF module learns corresponding image features of LiDAR points with a matched scale based on the image feature pyramid and attention mechanism for a better effect of fusion. Furthermore, based on the PPAF module, a new LiDAR-camera fusion-based 3D object detector named FS-Net is proposed, a two-stage detector with LiDAR voxel-based RPN and refinement network based on enriched LiDAR-camera features. Experiments on two public datasets demonstrate the effectiveness of our approach.

For multi-view multi-label learning, exploring the relations between multiple views and the rational utilization of sam-ple correlations are challenging tasks. In order to explic-itly explore the commonality and individuality between mul-tiple views and to make use of the sample correlations, Graph Convolution Network based Representation (GCNR) for multi-view multi-label learning is proposed. The model first extracts the intermediate representations and constructs the graphs based on different views. Then the intersection of different adjacency matrices and graph convolutional neu-ral networks are used to explore the commonality between different views, while the individuality of views is explored based on the same network parameters and respective adja-cency matrices. Finally, the common and individual represen-tations are concatenated to predict labels. The experimental results show that the proposed model outperforms the state-of-the-art methods on multiple datasets.

Driving scene data in the natural environments show an extremely imbalanced distribution, i.e. some scenes are in the majority while others are very rare. These rare driving scenes, also known as edge driving scenes, can dramatically affect the accuracy of driving scene recognition, as well as the safety, convenience, and intelligence of autonomous driving. Thus, this paper proposed the Driving Scene Supplementary Class Balancing Network to address edge driving scene recognition, which can effectively recognize edge driving scenes while training in imbalanced data. Our approach consists of three components. First, the Edge Scene Supplementer generates extremely rare scenes to supplement the training data to reduce the excessive imbalance between edge scenes and general scenes. Then, the Feature Encoder is responsible for efficiently extracting visual and temporal representations from the driving scenes. Finally, the Weight Balancing Classifier presents the classification results and re-balances the loss of driving scenes. To evaluate the performance of the proposed method, comprehensive experiments were performed on two datasets of real-world driving scenes. The results show that the proposed method outperforms the state-of-the-art methods in edge driving scene recognition.

Driving pattern recognition based on features, such as GPS, gear, and speed information, is essential to develop intelligent transportation systems. However, it is usually expensive and labor intensive to collect a large amount of labeled driving data from real-world driving scenes. The lack of a labeled data problem in a driving scene substantially hinders the driving pattern recognition accuracy. To handle the scarcity of labeled data, we have developed a novel discriminative transfer learning method for driving pattern recognition to leverage knowledge from related scenes with labeled data to improve recognition performance in unlabeled scenes. Note that data from different scenes may have different distributions, which is a major bottleneck limiting the performance of transfer learning. To address this issue, the proposed method adopts a discriminative distribution matching scheme with the aid of pseudolabels in unlabeled scenes. It is able to reduce the intraclass distribution disagreement for the same driving pattern among labeled and unlabeled scenes while increasing the interclass distance among different patterns. Pseudolabels in unlabeled scenes are updated iteratively via an ensemble strategy that preserves the data structure while enhancing the model robustness. To evaluate the performance of the proposed method, we conducted comprehensive experiments on real-world parking lot datasets. The results show that the proposed method can substantially outperform state-of-the-art methods in driving pattern recognition.

Recommender algorithms combining knowledge graph and graph convolutional network are becoming more and more popular recently. Specifically, attributes describing the items to be recommended are often used as additional information. These attributes along with items are highly interconnected, intrinsically forming a Knowledge Graph (KG). These algorithms use KGs as an auxiliary data source to alleviate the negative impact of data sparsity. However, these graph convolutional network based algorithms do not distinguish the importance of different neighbors of entities in the KG, and according to Pareto’s principle, the important neighbors only account for a small proportion. These traditional algorithms can not fully mine the useful information in the KG. To fully release the power of KGs for building recommender systems, we propose in this article KRAN, a Knowledge Refining Attention Network, which can subtly capture the characteristics of the KG and thus boost recommendation performance. We first introduce a traditional attention mechanism into the KG processing, making the knowledge extraction more targeted, and then propose a refining mechanism to improve the traditional attention mechanism to extract the knowledge in the KG more effectively. More precisely, KRAN is designed to use our proposed knowledge-refining attention mechanism to aggregate and obtain the representations of the entities (both attributes and items) in the KG. Our knowledge-refining attention mechanism first measures the relevance between an entity and it’s neighbors in the KG by attention coefficients, and then further refines the attention coefficients using a “richer-get-richer” principle, in order to focus on highly relevant neighbors while eliminating less relevant neighbors for noise reduction. In addition, for the item cold start problem, we propose KRAN-CD, a variant of KRAN, which further incorporates pre-trained KG embeddings to handle cold start items. Experiments show that KRAN and KRAN-CD consistently outperform state-of-the-art baselines across different settings.

Wi-Fi-based crowd counting is predominant because of its noninvasive and ubiquitous advantages. However, the existing Wi-Fi-based crowd counting systems have the constraint that there is always a maximum number of people counted. In order to address this issue, a Wi-Fi-based cross-environment crowd counting system, which has the capability of both estimating the walking direction and crowd counting by only one single link, called WiCrowd is proposed. WiCrowd relaxes the restriction of people number counted and demonstrates its extraordinary robustness when the environment changes. The signal change trends of the people flow are theoretically analyzed and people flow moving direction is inferred. The unique features using the eigenvalue of the convariance matrix of amplitude and phase are derived to effectively detect prominent signal changes led by the crowd movement near LoS. By adopting the augmented feature representations, the robustness of WiCrowd is improved when the environment changes. The experimental results in a typical indoor environment demonstrate the superior performance of WiCrowd. This system achieves 87.4%, 85.8%, and 79.4% recognition accuracy for the flow movement direction estimation, respectively, and 82.4% and 81.6% of the overall cross-environment accuracy for the number of subjects counted in the people flow.

Recent researches demonstrate that correlation modeling plays a key role in high-performance multi-label classification methods. However, existing methods do not take full advantage of correlation information, especially correlations in feature and label spaces of each image, which limits the performance of correlation-based multi-label classification methods. With more correlations considered, in this study, a Fully Associative Network (FAN) is proposed for fully exploiting correlation information, which involves both visual feature and label correlations. Specifically, FAN introduces a robust covariance pooling to summarize convolution features as global image representation for capturing feature correlation in the multi-label task. Moreover, it constructs an effective label correlation matrix based on a re-weighted scheme, which is fed into a graph convolution network for capturing label correlation. Then, correlation between covariance representations (i.e., feature correlation) and the outputs of GCN (i.e., label correlation) are modeled for final prediction. Experimental results on two datasets illustrate the effectiveness and efficiency of our proposed FAN compared with state-of-the-art methods.

Multiview clustering explores complementary information among distinct views to enhance clustering performance under the assumption that all samples have complete information in all available views. However, this assumption does not hold in many real applications, where the information of some samples in one or more views may be missing, leading to partial multiview clustering problems. In this case, significant performance degeneration is usually observed. A collection of partial multiview clustering algorithms has been proposed to address this issue and most treat all different views equally during clustering. In fact, because different views provide features collected from different angles/feature spaces, they might play different roles in the clustering process. With the diversity of different views considered, in this study, a novel adaptive method is proposed for partial multiview clustering by automatically adjusting the contributions of different views. The samples are divided into complete and incomplete sets, while a joint learning mechanism is established to facilitate the connection between them and thereby improve clustering performance. More specifically, the method is characterized by a joint optimization model comprising two terms. The first term mines the underlying cluster structure from both complete and incomplete samples by adaptively updating their importance in all available views. The second term is designed to group all data with the aid of the cluster structure modeled in the first term. These two terms seamlessly integrate the complementary information among multiple views and enhance the performance of partial multiview clustering. Experimental results on real-world datasets illustrate the effectiveness and efficiency of our proposed method.

Large labeled datasets are required for training a powerful semantic segmentation model. However, it is very expensive to construct pixel-wise annotated images. In this work, we propose a general batch mode active learning algorithm for semantic segmentation which automatically selects important samples to be labeled for building a competitive classifier. In our approach the edge information of an image is first introduced as a new selecting clue of active learning, which can measure the essential information relevant to segmentation performance. In addition, we also incorporate the informativeness based on Query by Committee (QBC) and representativeness criteria in our algorithm. We combine three clues to select a batch of samples during each iteration. It is shown that the image edge information is significant for the active learning for semantic segmentation in the experiments. And we also demonstrate the performance of our method outperforms the state of the art active learning approaches on the datasets of CamVid, Stanford Background and PASCAL VOC 2012.

Heterogeneous transfer clustering can translate knowledge from some related heterogeneous source domains to the target domain without any supervision. Existing works usually use a large amount of complete co-occurrence data to learn the projection functions mapping heterogeneous data to a common latent feature subspace. However, in many real applications, it is not practical to collect abundant co-occurrence data, while the available co-occurrence data are always incomplete. Another commonly encountered problem is that the complex structure of real heterogeneous data may result in substantial degeneration in clustering performance. To address these issues, we propose a heterogeneous transfer clustering method specifically designed for partial co-occurrence data (HTCPC). It is superior to the existing methods in three facets. First, HTCPC fully uses the partial co-occurrence data in both source and target domains to learn a latent space, maximally extracting useful knowledge for clustering from limited information. Second, it incorporates multi-layer hidden representations, accurately preserving the complex hierarchical structure of data. Third, it enforces approximately orthogonal constraint in representations, effectively characterizing the latent subspace with minimal redundancy. An efficient algorithm has been derived and implemented to realize the proposed HTCPC. A series of experiments on the real datasets have illustrated the advantage of the proposed approach compared with state-of-the-art methods.

Attributed graphs have attracted much attention in recent years. Different from conventional graphs, attributed graphs involve two different types of heterogeneous information, i.e., structural information, which represents the links between the nodes, and attribute information on each of the nodes. Clustering on attributed graphs usually requires the fusion of both types of information in order to identify meaningful clusters. However, most of existing works implement the combination of these two types of information in a "global" manner by treating all nodes equally and learning a global weight for the information fusion. To address this issue, this paper proposed a novel weighted K -means algorithm with "local" learning for attributed graph clustering, called adaptive fusion of structural and attribute information (Adapt-SA) and analyzed the convergence property of the algorithm. The key advantage of this model is to automatically balance the structural connections and attribute information of each node to learn a fusion weight, and get densely connected clusters with high attribute semantic similarity. Experimental study of weights on both synthetic and real-world data sets showed that the weights learned by Adapt-SA were reasonable, and they reflected which one of these two types of information was more important to decide the membership of a node. We also compared Adapt-SA with the state-of-the-art algorithms on the real-world networks with varieties of characteristics. The experimental results demonstrated that our method outperformed the other algorithms in partitioning an attributed graph into a community structure or other general structures.

Most existing metric learning methods focus on learning a similarity or distance measure relying on similar and dissimilar relations between sample pairs. However, pairs of samples cannot be simply identified as similar or dissimilar in many real-world applications, e.g., multi-label learning, label distribution learning or tasks with continuous decision values. To this end, in this paper we propose a novel relation alignment metric learning (RAML) formulation to handle the metric learning problem in those scenarios. Since the relation of two samples can be measured by the difference degree of the decision values, motivated by the consistency of the sample relations in the feature space and decision space, our proposed RAML utilizes the sample relations in the decision space to guide the metric learning in the feature space. Specifically, our RAML method formulates metric learning as a kernel regression problem, which can be efficiently optimized by the standard regression solvers. We carry out several experiments on the single-label classification, multi-label classification, and label distribution learning tasks, to demonstrate that our method achieves favorable performance against the state-of-the-art methods.

Multi-label problems arise in various domains, including automatic multimedia data categorization, and have generated significant interest in computer vision and machine learning community. However, existing methods do not adequately address two key challenges: exploiting correlations between labels and making up for the lack of labelled data or even missing labelled data. In this paper, we proposed to use a semi-supervised singular value decomposition (SVD) to handle these two challenges. The proposed model takes advantage of the nuclear norm regularization on the SVD to effectively capture the label correlations. Meanwhile, it introduces manifold regularization on mapping to capture the intrinsic structure among data, which provides a good way to reduce the required labelled data with improving the classification performance. Furthermore, we designed an efficient algorithm to solve the proposed model based on the alternating direction method of multipliers, and thus, it can efficiently deal with large-scale data sets. Experimental results for synthetic and real-world multimedia data sets demonstrate that the proposed method can exploit the label correlations and obtain promising and better label prediction results than the state-of-the-art methods.

One of the main research problems in heterogeneous transfer learning is to determine whether a given source domain is effective in transferring knowledge to a target domain, and then to determine how much of the knowledge should be transferred from a source domain to a target domain. The main objective of this paper is to solve this problem by evaluating the relatedness among given domains through transferred weights. We propose a novel method to learn such transferred weights with the aid of co-occurrence data, which contain the same set of instances but in different feature spaces. Because instances with the same category should have similar features, our method is to compute their principal components in each feature space such that co-occurrence data can be rerepresented by these principal components. The principal component coefficients from different feature spaces for the same instance in the co-occurrence data have the same order of significance for describing the category information. By using these principal component coefficients, the Markov Chain Monte Carlo method is employed to construct a directed cyclic network where each node is a domain and each edge weight is the conditional dependence from one domain to another domain. Here, the edge weight of the network can be employed as the transferred weight from a source domain to a target domain. The weight values can be taken as a prior for setting parameters in the existing heterogeneous transfer learning methods to control the amount of knowledge transferred from a source domain to a target domain. The experimental results on synthetic and real-world data sets are reported to illustrate the effectiveness of the proposed method that can capture strong or weak relations among feature spaces, and enhance the learning performance of heterogeneous transfer learning.

Multi-label problems arise in various domains including automatic multimedia data categorization, and have generated significant interest in computer vision and machine learning community. However, existing methods do not adequately address two key challenges: exploiting correlations between labels and making up for the lack of labeled data or even missing labels. In this paper, we proposed a semi-supervised low-rank mapping (SLRM) model to handle these two challenges. SLRM model takes advantage of the nuclear norm regularization on mapping to effectively capture the label correlations. Meanwhile, it introduces manifold regularizer on mapping to capture the intrinsic structure among data, which provides a good way to reduce the required labeled data with improving the classification performance. Furthermore, we designed an efficient algorithm to solve SLRM model based on alternating direction method of multipliers and thus it can efficiently deal with large-scale datasets. Experiments on four real-world multimedia datasets demonstrate that the proposed method can exploit the label correlations and obtain promising and better label prediction results than state-of-the-art methods.

Heterogeneous transfer learning has recently gained much attention as a new machine learning paradigm in which the knowledge can be transferred from source domains to target domains in different feature spaces. Existing works usually assume that source domains can provide accurate and useful knowledge to be transferred to target domains for learning. In practice, there may be noise appearing in given source (text) and target (image) domains data, and thus, the performance of transfer learning can be seriously degraded. In this paper, we propose a robust and non-negative collective matrix factorization model to handle noise in text-to-image transfer learning, and make a reliable bridge to transfer accurate and useful knowledge from the text domain to the image domain. The proposed matrix factorization model can be solved by an efficient iterative method, and the convergence of the iterative method can be shown. Extensive experiments on real data sets suggest that the proposed model is able to effectively perform transfer learning in noisy text and image domains, and it is superior to the popular existing methods for text-to-image transfer learning.