Unsupervised graph domain adaptation (UGDA) aims to transfer knowledge from a labeled source graph to an unlabeled target graph, addressing the performance degradation caused by distributional shifts in node attributes and graph structures across domains. Despite recent progress, existing UGDA approaches still face two key challenges: (C1) Data-level: Most methods rely on a single source domain, overlooking the complementary knowledge that could be leveraged from multiple sources. (C2) Model-level: Many UGDA models emphasize complex, handcrafted Graph neural network (GNN) architectures, while simpler yet effective designs with propagation (P) & transformation (T) pipeline remain underexplored. To address these challenges, in this paper, we propose a novel approach, which leverages Concise Propagation–Transformation pipeline for multi-source unsupervised Graph Domain Adaptation, dubbed as CPT-GDA, to better capture complementary knowledge from multiple sources in an efficient manner. Specifically, the proposed CPT-GDA adopts a dual-branch GNN architecture with different depths of propagation but the same P-T patterns, which enables the model to efficiently learn node representations to mitigate domain discrepancy. Meanwhile, to facilitate effective knowledge transfer across graphs, we derive three optimization objectives: (1) the classifier loss to learn discriminative representations; (2) the alignment loss weighted by the graph Wasserstein distance to align the structure and feature distribution; and (3) the pseudo-label loss to refine target node representations. Extensive experiments on real-world datasets confirm that the proposed method outperforms recent state-of-the-art baselines, demonstrating its effectiveness.

As privacy protection gains momentum, federated learning has emerged as a cutting-edge approach in medical image analysis. However, the intricacies of medical image segmentation task have led to a dearth of research in this domain, with existing studies falling short in tackling two pivotal challenges: The traditional model with the uniform global model underperforms for certain clients due to the heterogeneity and non-Independent Identically Distributed(non-IID) data across medical institutions. And the communication between the server and clients often incurs significant time costs. This paper introduces a novel Personalized Asynchronous Federated learning for Medical Image Segmentation model, dubbed PAFedMIS, to mitigate the negative impact of the heterogeneous data and fully utilized the waiting time, in medical image segmentation. Comprehensive experiments on ISIC2018 demonstrate the enhanced model accuracy and training efficiency of PAFedMIS.

Federated learning (FL) is a machine learning approach where multiple participants train a model together without sharing their privacy data. The challenge of long-tail data heterogeneity in FL causes imbalanced data distribution among clients and significant disparities in data quantities across different classes. To tackle the long-tail distribution in FL, in this paper, we draw inspiration from the equiangular tight frame to establish a fixed balanced classifier, enhancing the model’s generalization ability for classes with fewer instances. Therefore, feature extractors and a loss aligned with fixed classifiers are designed to address the long-tail distribution in FL. The algorithm also utilizes Sample Standardization and Batch Normalization to standardize the feature space of samples, preventing the impact of the magnitude difference of features across different classes on the prediction probability. In addition, we also propose a logit adjustment loss under a mixed low-temperature setting. Experiments demonstrate that our algorithm outperforms state-of-the-art methods in FL settings with long-tail distribution.

Style transfer is a challenging task in computer vision, aiming to blend the stylistic features of one image with the content of another while preserving the content details. Traditional methods often face challenges in terms of computational efficiency and fine-grained content preservation. In this paper, we propose a novel feature modulation mechanism based on parameterized normalization, where the modulation parameters for content and style features are learned using a dual convolution network (BiConv). These parameters adjust the mean and standard deviation of the features, improving both the stability and quality of the style transfer process. To achieve fast inference, we introduce an efficient acceleration technique by leveraging a row and column weighted attention matrix. In addition, we incorporate a contrastive learning scheme to align the local features of the content and the stylized images, improving the fidelity of the generated output. Experimental results demonstrate that our method significantly improves the inference speed and the quality of style transfer while preserving content details, outperforming existing approaches based on both convolution and diffusion.

Distribution shifts in recommender systems between training and testing in user-item interactions lead to inaccurate recommendations. Despite the promising performance of test-time adaptation technology in various domains, it still faces challenges in recommender systems due to the impracticality of fine-tuning models and the infeasibility of obtaining test-time labels. To address these challenges, we first propose a Test-Time Adaptation framework for Graph-based Recommender system, named TTA-GREC, to dynamically adapt user-item graphs at test time in a data-centric way, handling distribution shifts effectively. Specifically, our TTA-GREC targets KG-enhanced GNN-based recommender systems with three core components: (1) Pseudo-label guided UI graph transformation for adaptive improvement; (2) Rationale score guided KG graph revision for semantic enhancement; and (3) Sampling-based self-supervised adaptation for contrastive learning. Experiments demonstrate TTA-GREC's superiority at test time and provide new data-centric insights on test-time adaptation for better recommender system inference.

While impressive performance has been achieved in image captioning, the limited diversity of the generated captions and the large parameter scale remain major barriers to the real-word application of these systems. In this work, we propose a lightweight image captioning network in combination with continuous diffusion, called Prefix-diffusion. To achieve diversity, we design an efficient method that injects prefix image embeddings into the denoising process of the diffusion model. In order to reduce trainable parameters, we employ a pre-trained model to extract image features and further design an extra mapping network. Prefix-diffusion is able to generate diverse captions with relatively less parameters, while maintaining the fluency and relevance of the captions benefiting from the generative capabilities of the diffusion model. Our work paves the way for scaling up diffusion models for image captioning, and achieves promising performance compared with recent approaches.

Due to the limitation of spatial receptive field, it is challenging for CNN-based style transfer methods to capture rich and long-range semantic concepts in artworks. Though the transformer provides a fresh solution by considering long-range dependencies, it suffers from the heavy burdens of parameter scale and computation cost especially in vision tasks. In this paper, we design a compact transformer architecture AdaFormer to address the problem. The model scale shrinks about 20% compared to state-of-the-art transformer for style transfer. Furthermore, we explore the adaptive style transfer by letting the content to select the detailed style element automatically and adaptively, which encourages the output to be both appealing and reasonable. We evaluate AdaFormer comprehensively in the experiments and the results have shown the effectiveness and superiority of our approach compared to existing artistic methods. Diverse plausible stylized images are obtained with better content preservation, more convincing style sfumato and lower computation complexity.

With the rise of neural networks, many deep hashing networks have been successfully trained on the basis of large-scale data. However, the conventional learning process has received increasing challenges from the data privacy concerns and the decentralized storage status, especially in sensitive scenarios like surveillance retrieval. Further considering the probable different distributions of the decentralized data, in this paper, we present a collaborative hashing paradigm FedA-Hash (Federating Adapted Hashing nets) to produce personalized hashing models for the clients without exchanging their local data. To this end, the bilateral knowledge is blended gradually during the learning process between the aggregated global model and the local hashing model, instead of replacing the local model with the global model directly. Extensive experiments are conducted on representative hashing networks, involving tasks as image retrieval and person re-identification. The results show that FedA-Hash significantly enables the collaborated performance among different clients.

Though existing researches have achieved impressive results in controlled text generation, they focus mainly on single-attribute control. However, in applications like automatic comments, the topic and sentiment need to be controlled simultaneously. In this work, we propose a new framework for multi-attribute controlled text generation. To achieve this, we design a contrastive-generator that can effectively generate texts with more attributes. In order to increase the convergence of the text on the desired attributes, we adopt an external-discriminator to distinguish whether the generated text holds the desired attributes. Moreover, we propose top-n weighted decoding to further improve the relevance of texts to attributes. Automated evaluations and human evaluations show that our framework achieves remarkable controllability in multi-attribute generation while keeping the text fluent and diverse. It also yields promising performance on zero-shot generation.

Style transfer has been well studied in recent years with excellent performance processed. While existing methods usually choose CNNs as the powerful tool to accomplish superb stylization, less attention was paid to the latent style space. Rare exploration of underlying dimensions results in the poor style controllability and the limited practical application. In this work, we rethink the internal meaning of style features, further proposing a novel unsupervised algorithm for style discovery and achieving personalized manip-ulation. In particular, we take a closer look into the mechanism of style transfer and obtain different artistic style components from the latent space consisting of different style features. Then fresh styles can be generated by linear combination according to various style components. Experimental results have shown that our approach is superb in 1) restylizing the original output with the diverse artistic styles discovered from the latent space while keeping the content unchanged, and 2) being generic and compatible for various style transfer methods. Our code is available in this page: https://github.com/Shelsin/ArtIns.

Recent studies highlight the vulnerability of convolutional neural networks (CNNs) to adversarial attacks, which also calls into question the reliability of forensic methods. Existing adversarial attacks generate one-to-one noise, which means these methods have not learned the fingerprint information. Therefore, we introduce two powerful attacks, fingerprint copy-move attack, and joint feature-based auto-learning attack. To validate the performance of attack methods, we move a step ahead and introduce the higher possible defense mechanism relation mismatch. which expands the characterization differences of classifiers in the same classification network. Extensive experiments show that relation mismatch is superior in recognizing adversarial examples and prove that the proposed fingerprint-based attacks are more powerful. Both proposed attacks show excellent attack transferability to unknown samples. The Pytorch® implementations of these methods can download from an open-source GitHub projecthttps://github.com/Dlut-lab-zmn/Source-attack.

Considering that traditional image steganography technologies suffer from the potential risk of failure under lossy channels, an enhanced adaptive steganography with multiple robustness against image processing attacks is proposed, while maintaining good detection resistance. First, a robust domain constructing method is proposed utilizing robust element extraction and optimal element modification, which can be applied to both spatial and JPEG images. Then, a robust steganography is proposed based on “Robust Domain Constructing + RS-STC Codes,” combined with cover selection, robust cover extraction, message coding, and embedding with minimized costs. In addition, to provide a theoretical basis for message extraction integrity, the fault tolerance of the proposed algorithm is deduced using error model based on burst errors and decoding damage. Finally, on the basis of parameter discussion about robust domain construction, performance experiments are conducted, and the recommended coding parameters are given for lossy channels with different attacks using the analytic results for fault tolerance. A series of experimental results demonstrate that the proposed algorithm can extract embedded messages with significantly higher accuracy after different attacks, such as compression, noising, scaling and other attacks, compared with the state-of-the-art adaptive steganography, and robust watermarking algorithms, while maintaining good detection resistant performance.

Semantic image segmentation can be accomplished by assigning a proper object category label to each meaningful region of an image. Beyond the original bottom-up models, the use of top-down categorization information has been applied to semantic segmentation to improve performance. An excellent example of such a top-down scheme is to integrate a Conditional Random Field (CRF) model with sparse dictionary learning. However, the existing solutions merely consider the discrimination of dictionaries to obtain better sparse codes, without considering the inherent data locality characteristics. In this paper, we explore such characteristics and propose a novel semantic segmentation framework based on an innovative CRF model with locality-consistent dictionary learning. In particular, we propose two new locality-consistent dictionary learning strategies by capturing the local consistencies in the feature space and the label space. In addition, we develop a joint dictionary and a CRF model parameter learning algorithm to seamlessly integrate the proposed locality-consistent dictionary learning strategies into the CRF model. Extensive experiments are conducted with two popular databases of different traits (i.e., Graz-02 and PASCAL-CONTEXT). The simulation results confirm the efficiency of the proposed scheme, especially when training data are limited.

For many real-world multimedia applications, data are often described by multiple views. Therefore, multi-view learning researches are of great significance. Traditional multi-view clustering methods assume that each view has complete data. However, missing data or partial data are more common in real tasks, which results in partial multi-view learning. Therefore, we propose a novel multi-view clustering method, called Partial Multi-view Subspace Clustering (PMSC), to address the partial multi-view problem. Unlike most existing partial multi-view clustering methods that only learn a new representation of the original data, our method seeks the latent space and performs data reconstruction simultaneously to learn the subspace representation. The learned subspace representation can reveal the underlying subspace structure embedded in original data, leading to a more comprehensive data description. In addition, we enforce the subspace representation to be non-negative, yielding an intuitive weight interpretation among different data. The proposed method can be optimized by the Augmented Lagrange Multiplier (ALM) algorithm. Experiments on one synthetic dataset and four benchmark datasets validate the effectiveness of PMSC under the partial multi-view scenario.

In realistic steganalysis applications, the mismatched problem can lead to the degradation of performance in steganalysis. The main reason is the discrepancy of feature distributions between training set and testing set. In this paper, we present a Contribution-based Feature Transfer (CFT) algorithm for JPEG mismatched steganalysis. CFT tries to learn two transformations to transfer training set features by evaluating both the sample feature and dimensional feature contributions. We can obtain new feature representations so as to approach the feature distribution of the testing samples. The comparison to prior arts reveals the superiority of CFT on the experiments for the mismatched JPEG steganalysis in the heterogeneous cover source scenario.

Saliency detection has been long researched. However, most existing algorithms can not uniformly highlight salient objects. To approach this problem, we propose a novel saliency detection algorithm based on the Local Single Gaussian Model (LSGM). First, we utilize a bottom-up model to generate an initial saliency map and construct a background dictionary and a foreground dictionary based on the initial saliency map, respectively. Then, a LSGM is used to obtain a LSGM-based map. Note that we construct a corresponding LSGM for each superpixel region and thus the LSGM is a dynamic model with geometric structure information. Finally, we integrate the LSGM-based saliency map and the initial bottom-up map with global information as the final saliency map. Extensive experiments on four public datasets show that our algorithm outperforms state-of-the-art methods.

Unsupervised feature selection has shown significant potential in distance-based clustering tasks. This paper proposes a novel triplet induced method. Firstly, a triplet-based loss function is introduced to enforce the selected feature groups to preserve ordinal locality of original data, which contributes to distance-based clustering tasks. Secondly, we simplify the orthogonal basis clustering by imposing an orthogonal constraint on the feature projection matrix. Consequently, a general framework for simultaneous feature selection and clustering is discussed. Thirdly, an alternating minimization algorithm is employed to efficiently optimize the proposed model together with rapid convergence. Extensive comparison experiments on several benchmark datasets well validate the encouraging gain in clustering from our proposed method.

In this letter, we propose a novel classification strategy called the coupled dictionary learning for target recognition in synthetic aperture radar (SAR) images. First, we train structured synthesis dictionaries to reflect the difference among each category. Second, we introduce a shared dictionary to reduce the effect of common features, such as the high similarity caused by specular reflection. Finally, we use the analysis dictionary to improve the efficiency of recognition by eliminating the constraint of the l0-norm or l1-norm of sparse code. Experimental results on Moving and Stationary Target Acquisition and Recognition data set indicate that our method can achieve better performance in SAR target recognition than the state-of-the-art methods, such as tritask joint sparse representation and CKLR. Especially, this method can be more robust when the depressions have obvious changes.

The CAPTCHA has become an important issue in multimedia security. Aimed at a commonly used text-based CAPTCHA, this paper outlines some typical methods and summarizes the technological progress in text-based CAPTCHA breaking. First, the paper presents a comprehensive review of recent developments in the text-based CAPTCHA breaking field. Second, a framework of text-based CAPTCHA breaking technique is proposed. And the framework mainly consists of preprocessing, segmentation, combination, recognition, postprocessing, and other modules. Third, the research progress of the technique involved in each module is introduced, and some typical methods of segmentation and recognition are compared and analyzed. Lastly, the paper discusses some problems worth further research.

Dictionary learning (DL) plays an important role in pattern classification. However, learning a discriminative dictionary has not been well addressed in analysis dictionary learning (ADL). This letter proposes a Class-aware Analysis Dictionary Learning (CADL) model to improve the classification performance of conventional ADL. The objective function of CADL mainly includes two parts to promote the discriminability. The first part aims to learn a discriminative analysis subdictionary for each class instead of a global dictionary for all classes. The learned analysis dictionary is class-aware, generating a block-diagonal coding coefficient matrix. The second part aims to enhance the discrimination of coding coefficients by integrating a max-margin regularization term into our proposed framework. This term ensures the coefficients of different classes to be separated by a max-margin, which boosts the confidence of classification. A theoretical analysis is also given to support the max-margin regularization term from the perspective of preserving the pairwise relations of samples in coding space. We employ an alternating minimization algorithm to iteratively find the convergent solution. By evaluating our method on four pattern classification datasets, we demonstrate the superiority of our CADL method to the state-of-the-art DL methods.

The task of semantic segmentation is to infer a predefined category label for each pixel in the image. For most cases, image segmentation is established as a fully supervised task. These methods all built on the basis of having access to sufficient pixel-wise annotated samples for training. However, obtaining the satisfied ground truth is not only labor intensive but also time-consuming, which severely hinders the generality of these fully supervised methods. Instead of pixel-level ground truth, weakly supervised approaches learn their models from much less prior information, e.g., image-level annotation. In this paper, we propose a novel conditional random field (CRF) based framework for weakly supervised semantic segmentation. Enlightened by jigsaw puzzles, we start the approach with merging superpixels from an image into larger pieces by a newly designed strategy. Then pieces from all the training images are gathered and associated with appropriate semantic labels by CRF. Thus, the piece library is constructed, achieving remarkable universality and flexibility. In the case of testing, we compare the superpixels with image pieces in the library and assign them the labels that minimize the potential energy. In addition, the proposed framework is fit for domain adaption and obtains promising results, which is of great practical value. Extensive experimental results on PASCAL VOC 2007, MSRC-21, and VOC 2012 databases demonstrate that our framework outperforms or is comparable to state-of-the-art segmentation methods.

Dictionary learning (DL) has been successfully applied to various pattern classification tasks in recent years. However, analysis dictionary learning (ADL), as a major branch of DL, has not yet been fully exploited in classification due to its poor discriminability. This paper presents a novel DL method, namely Discriminative Analysis Dictionary Learning (DADL), to improve the classification performance of ADL. First, a code consistent term is integrated into the basic analysis model to improve discriminability. Second, a triplet constraint-based local topology preserving loss function is introduced to capture the discriminative geometrical structures embedded in data. Third, correntropy induced metric is employed as a robust measure to better control outliers for classification. Then, half-quadratic minimization and alternate search strategy are used to speed up the optimization process so that there exist closed-form solutions in each alternating minimization stage. Experiments on several commonly used databases show that our proposed method not only significantly improves the discriminative ability of ADL, but also outperforms state-of-the-art synthesis DL methods.

In recent years, dictionary learning (DL) has shown significant potential in various classification tasks. However, most of previous works aim to learn a synthesis dictionary. The other major category of DL-analysis dictionary learning has not been fully exploited yet. This paper proposes a novel DL method, named Topology Preserving Dictionary Learning (TPDL). First, we propose a triplet-constraint-based topology preserving loss function to capture the underlying local topological structures of data in a supervised manner. Second, a sparse-label-matrix-based function is integrated into the basic analysis model to improve discriminative ability. Third, Huber M-estimator is employed as a robust metric to handle the errors (e.g., outliers and noise) that possibly exist in data. Then, an alternating optimization algorithm is developed based on half-quadratic minimization and alternate search strategy. Closed-form solutions in each alternating optimization stage speed up the minimization process. Experiments on four commonly used datasets show that our proposed TPDL achieves competitive performance in contrast to state-of-the-art DL methods.

In this study, the authors consider additive spread‐spectrum (SS) data embedding in transform‐domain host data. Conventional additive SS embedding schemes use an equal‐amplitude modulated carrier to deposit one information symbol across a group of host data coefficients which act as interference to SS signal of interest. If there is a flexibility of assigning different amplitudes across symbol bits, the probability of error can be further reduced by adaptively allocating amplitude to each symbol bit based on its own host/interference. In this study, they present a novel amplitude‐adaptive SS embedding scheme. Particularly, symbol‐by‐symbol adaptive amplitude allocation algorithms are developed to compensate for the impact from the known interference. They aim at designing the SS embedding amplitude for each symbol adaptively in order to minimise the receiver bit‐error‐rate (BER) at any given distortion level. Then, optimised amplitude allocation for multi‐carrier/multi‐message embedding in the same host data is studied as well. Finally, they consider the problem of amplitude optimisation for an ideal scenario where no external noise is introduced during embedding and transmission. Extensive experimental results illustrate that the proposed amplitude‐adaptive SS embedding scheme can provide order‐of‐magnitude performance improvement over several other state‐of‐the‐art SS embedding schemes.

Discriminative dictionary learning (DDL) has been applied to various pattern classification problems. Despite satisfying experimental results, most existing discriminative dictionary learning methods emphasize too much on the role of l0or l1-norm sparsity, while the underlying local structure of original data is totally ignored. In this paper, we present a novel dictionary learning method, named Locality Sensitive Discriminative Dictionary Learning (LSDDL), which combines basic dictionary learning scheme and locality relationship of original data which is propagated to the coding vectors. The learned discriminative dictionary can map the original data points into a new space in which the nearby points with the same label are close to each other while the nearby points with different labels are far apart. Experiments clearly show that our method has very competitive performance in contrast to previous discriminative dictionary learning methods.

Fine-grained visual categorization (FGVC) refers to the task of classifying objects that belong to the same basic-level class (e.g., different bird species). Since the subtle inter-class variation often exists on small parts (e.g., beak, belly, etc.), it is reasonable to localize semantic parts of an object before describing it. However, unsupervised part-segmentation methods often suffer from over-segmentation which harms the quality of image representation. In this paper, we present a fine-tuning approach to tackle this problem. To this end, we perform a greedy algorithm to optimize an intuitive objective function, preserving principal parts meanwhile filtering noises, and further construct mid-level parts beyond the refined parts toward a more descriptive representation. Experiments demonstrate that our approach achieves competitive classification accuracy on the CUB-200-2011 dataset with both Fisher vectors and deep conv-net features.

Due to its compact binary codes and efficient search scheme, image hashing method is suitable for large-scale image retrieval. In image hashing methods, Hamming distance is used to measure similarity between two points. For K-bit binary codes, the Hamming distance is an into and bounded by K. Therefore, there are many returned images share the same Hamming distances with the query. In this paper, we propose an efficient image ranking method based on bit importance of binary code. Compared with the returned images of Hamming distance, important bits of query image are detected. Then, large weights are assigned to important bits and small weights are assigned to minor bits. The advantage of this proposed method is calculation efficiency. Evaluations on two large-scale image data sets demonstrate the efficacy of our binary code ranking method based on bit importance.

Most universal JPEG steganalysis approaches rely on the assumption that training and testing samples come from the same distribution. They fail when training set and testing set are mismatched. In this paper, we propose generalized transfer component analysis for mismatched JPEG steganalysis to derive new representations from original features for training and testing samples to correct the mismatches. We first apply domain alignment to transform source domain (training set) to an intermediate domain closer to target domain (testing set). Then a set of common transfer components are learnt across two domains by minimizing the distribution distance between them. In the space spanned by these transfer components, two domains manifest similar characteristics and preserve enough discrimination to different categories. Extensive experiments demonstrate our method performs well in mismatched JPEG steganalysis.

Cell-phones have become a necessary communication accessory in daily life. MMS (Multimedia Messaging Service) used by smart phones has caused higher requirement on mobile image manipulation. Classifying image source cell-phones has become a major issue in the cell-phone communication forensics. There are two ways usually used for tracing and identifying the source device: image characteristics and equipment fingerprint. Both of the above schemes require a set of images captured by known source cell-phones for training a classification model. To avoid using any prior knowledge in practical scenarios, a graph based approach was proposed to classify the source cell-phones. Though an acceptable result has been obtained, a problem of incomplete classification appears in the case that one image is classified wrong into a single subset. In this paper, a silhouette coefficient based algorithm is proposed for source cell-phone classification. The spectral clustering algorithm is adopted in graph partitioning and the silhouette coefficient is used to extract the optimal classification from all the possibilities of classification. Experimental results show the validity of the proposed method.

This paper proposes a novel robust sparse representation method, called the two-stage sparse representation (TSR), for robust recognition on a large-scale database. Based on the divide and conquer strategy, TSR divides the procedure of robust recognition into outlier detection stage and recognition stage. In the first stage, a weighted linear regression is used to learn a metric in which noise and outliers in image pixels are detected. In the second stage, based on the learnt metric, the large-scale dataset is firstly filtered into a small set according to the nearest neighbor criterion. Then a sparse representation is computed by the non-negative least squares technique. The sparse solution is unique and can be optimized efficiently. The extensive numerical experiments on several public databases demonstrate that the proposed TSR approach generally obtains better classification accuracy than the state of the art Sparse Representation Classification (SRC). At the same time, by using the TSR, a significant reduction of computational cost is reached by over fifty times in comparison with the SRC, which enables the TSR to be deployed more suitably for large-scale dataset.