Data-Centric Sequential Recommendation (DaCSR) has emerged as a promising technique that enhances dataset quality to better capture user preferences without increasing training complexity. However, mining item relations to improve data quality remains challenging due to the intricate nature of interaction sequences. Existing methods predominantly either: 1) optimize models to learn such item relations from fixed datasets at significant training cost, or 2) employ generative models to adaptively learn only interaction patterns, which lack interpretability and cannot guarantee effective data quality enhancement. In this paper, we pioneer a relation-guided dataset augmentation and regeneration framework for sequential recommendation called \textbf{RaSR}. This framework can significantly improve model performance on original datasets while maintaining training efficiency without modifying the model architecture. Specifically, we first preprocess user interactions to construct standardized sequential data and extract semantic representations via a Large Language Model (LLM). We then build a multi-relation graph with manually predefined metrics and semantic representations to generate augmented datasets. Finally, a relation-aware generator can produce regenerated datasets with both the multi-relation graph and the augmented dataset. To verify the effectiveness of RaSR, we conduct experiments on various backbone models and datasets, and achieve significant performance improvement compared to training the model only on the original dataset.

Fake orders pose increasing threats to sequential recommender systems by misleading recommendation results through artificially manipulated interactions, including click farming, context-irrelevant substitutions, and sequential perturbations. Unlike injecting carefully designed fake users to influence recommendation performance, fake orders embedded within genuine user sequences aim to disrupt user preferences and mislead recommendation results, thereby manipulating exposure rates of specific items to gain competitive advantages. To protect users' authentic interest preferences and eliminate misleading information, this paper aims to perform precise and efficient rectification on compromised sequential recommender systems while avoiding the enormous computational and time costs of retraining existing models. Specifically, we identify that fake orders are not absolutely harmful—in certain cases, partial fake orders can even have a data augmentation effect. Based on this insight, we propose Dual-view Identification and Targeted Rectification (DITaR), which primarily identifies harmful samples to achieve unbiased rectification of the system. The core idea of this method is to obtain differentiated representations from collaborative and semantic views for precise detection, and then filters detected suspicious fake orders to select truly harmful ones for targeted rectification with gradient ascent. This ensures that useful information in fake orders is not removed while preventing bias residue. Moreover, it maintains the original data volume and sequence structure, thus protecting system performance and trustworthiness to achieve optimal unbiased rectification. Extensive experiments on three datasets demonstrate that DITaR achieves superior performance compared to state-of-the-art methods in terms of recommendation quality, computational efficiency, and system robustness.

Xiaoquan Yi, Haixing Wu, Haozhao Wang, Yichen Li, Yuhua Li, Rui Zhang, Ruixuan Li. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Sequential recommendation models have been widely adopted for modeling user behavior. Existing approaches typically construct user interaction sequences by sorting items according to timestamps and then model user preferences from historical behaviors. While effective, such a process only considers the order of temporal information but overlooks the actual time spans between interactions, resulting in a coarse representation of users' temporal dynamics and limiting the model's ability to capture long-term and short-term interest evolution. To address this limitation, we propose RoTE, a novel multi-level temporal embedding module that explicitly models time span information in sequential recommendation. RoTE decomposes each interaction timestamp into multiple temporal granularities, ranging from coarse to fine, and incorporates the resulting temporal representations into item embeddings. This design enables models to capture heterogeneous temporal patterns and better perceive temporal distances among user interactions during sequence modeling. RoTE is a lightweight, plug-and-play module that can be seamlessly integrated into existing Transformer-based sequential recommendation models without modifying their backbone architectures. We apply RoTE to several representative models and conduct extensive experiments on three public benchmarks. Experimental results demonstrate that RoTE consistently enhances the corresponding backbone models, achieving up to a 20.11% improvement in NDCG@5, which confirms the effectiveness and generality of the proposed approach. Our code is available at https://github.com/XiaoLongtaoo/RoTE.

Web platforms are reshaping resource allocation in distributed energy networks globally, from off-grid communities to lunar bases. Algorithmic decision-makers face the fundamental challenge of fairly distributing scarce resources among heterogeneous stakeholders. Traditional approaches assume complete rationality with perfect information and unlimited computation, yet distributed networks only permit local observation, requiring fairness to emerge from individual strategic interactions. Centralized optimization fails due to exponential complexity, rule-based methods cannot adapt to disruptions, and existing platforms translate economic inequality into energy access inequality. Recognizing the unattainability of complete rationality necessitates bounded rationality: pursuing provably convergent satisficing solutions under incomplete information and limited computation, translating natural language ethics into computable constraints, and designing incentives so self-interested behavior satisfies fairness at equilibrium. We propose a unified semantic-game-distributed framework. Large language models map ambiguous ethical principles into game-theoretic parameters through semantic parameterization, with contrastive learning ensuring semantic consistency and temporal stability. A two-layer Stackelberg game implements incentive design: the platform signals through differentiated pricing while nodes optimize locally, enabling fairness to emerge from equilibrium. Distributed asynchronous iteration achieves global convergence through local communication, with cognitive models adaptively adjusting step sizes and differential perturbation preserving privacy. Theoretical analysis establishes equilibrium existence and convergence guarantees, while extreme scenarios validate robustness under information scarcity and high uncertainty.

The rapid growth of cloud-based image storage has made privacy-preserving content-based image retrieval increasingly critical for secure and efficient image search. Existing schemes, however, face three key challenges: limited verifiability of search results, inefficient and fault-intolerant support for dynamic updates, and unintended leakage of similarity information. In this paper, we proposeV-FDEIS, a fault-tolerant verifiable dynamic encrypted image search framework for multi-user cloud environments. V-FDEIS simultaneously supports verifiable similarity computation against potentially malicious cloud servers, fault-tolerant dynamic updates in the presence of erroneous client operations, and efficient multi-user search while effectively concealing similarity relationships. A threshold-based reproof mechanism, combined with aggregate verification, ensures low verification overhead even under frequent updates. Extensive theoretical analysis and experimental evaluation demonstrate that V-FDEIS achieves strong privacy guarantees, correctness, fault tolerance, and practical efficiency, outperforming existing approaches.

Graph contrastive learning (GCL) is a powerful self-supervised learning approach. However, existing GCL methods are designed for homophilic graphs, using low-pass filters that struggle to capture high-frequency components in heterophilic graphs. We proposeGraphContrastiveLearning withRegularization and stabilization techniques enhanced high-passFilter (GCLRF).REgularization andStabilization techniques enhancedHigh-pass filter (RESH) can serve as a mutually promoting plug-in, significantly improving the performance of various homophilic GCL training strategies on heterophilic graphs. We also investigate four component orderings in RESH and identify the optimal fusion mechanism, demonstrating its critical impact on performance. Experiments show GCLRF achieves state-of-the-art (SOTA) performance across six benchmark datasets in node classification and clustering. Notably, on the Cornell dataset, GCLRF outperformers classification accuracy by 6.76% and achieves a 23.64%relative improvement in clustering normalized mutual information (NMI).

With the rise of generative paradigms, generative recommendation has garnered increasing attention. The core component is the item code , generally derived by quantizing collaborative or semantic representations to serve as candidate items identifiers in the context. However, existing methods typically construct separate codes for each modality, leading to higher computational and storage costs and hindering the integration of their complementary strengths. Considering this limitation, we seek to integrate two different modalities into a unified code , fully unleashing the potential of complementary nature among modalities. Nevertheless, the integration remains challenging: the integrated embedding obtained by the common concatenation method would lead to underutilization of collaborative knowledge, thereby resulting in limited effectiveness. To address this, we propose a novel method, named UNGER, which integrates semantic and collaborative knowledge into a UNified code for Generative Recommendation . Specifically, we propose to adaptively learn an integrated embedding through the joint optimization of cross-modality knowledge alignment and next-item prediction tasks. Subsequently, to mitigate the information loss caused by the quantization process, we introduce an intra-modality knowledge distillation task, using the integrated embeddings as supervised signals to compensate. Extensive experiments on three widely used benchmarks demonstrate the superiority of our approach compared to existing methods.

Cross-Domain Few-Shot Learning (CDFSL) requires the model to transfer knowledge from the data-abundant source domain to data-scarce target domains for fast adaptation, where the large domain gap makes CDFSL a challenging problem. Masked Autoencoder (MAE) excels in effectively using unlabeled data and learning image’s global structures, enhancing model generalization and robustness. However, in the CDFSL task with significant domain shifts, we find MAE even shows lower performance than the baseline supervised models. In this paper, we first delve into this phenomenon for an interpretation. We find that MAE tends to focus on low-level domain information during reconstructing pixels while changing the reconstruction target to token features could mitigate this problem. However, not all features are beneficial, as we then find reconstructing high-level features can hardly improve the model’s transferability, indicating a trade-off between filtering domain information and preserving the image’s global structure. In all, the reconstruction target matters for the CDFSL task. Based on the above findings and interpretations, we further propose Domain-Agnostic Masked Image Modeling (DAMIM) for the CDFSL task. DAMIM includes an Aggregated Feature Reconstruction module to automatically aggregate features for reconstruction, with balanced learning of domain-agnostic information and images’ global structure, and a Lightweight Decoder module to further benefit the encoder’s generalizability. Experiments on four CDFSL datasets demonstrate that our method achieves state-of-the-art performance.

Cross-Domain Few-Shot Segmentation (CDFSS) is proposed to transfer the pixel-level segmentation capabilities learned from large-scale source-domain datasets to downstream target-domain datasets, with only a few annotated images per class. In this paper, we focus on a well-observed but under-explored phenomenon in CDFSS: for target domains, particularly those distant from the source domain, segmentation performance peaks at the very early epochs, and declines sharply as the source-domain training proceeds. We delve into this phenomenon for an interpretation: low-level features are vulnerable to domain shifts, leading to sharper loss landscapes during the source-domain training, which is the devil of CDFSS. Based on this phenomenon and interpretation, we further propose a method that includes two plug-and-play modules: one to flatten the loss landscapes for low-level features during source-domain training as a novel sharpness-aware minimization method, and the other to directly supplement target-domain information to the model during target-domain testing by low-level-based calibration. Extensive experiments on four target datasets validate our rationale and demonstrate that our method surpasses the state-of-the-art method in CDFSS signifcantly by 3.71% and 5.34% average MIoU in 1-shot and 5-shot scenarios, respectively.

Long-term time series forecasting (LTSF) seeks to make accurate long-term predictions by leveraging extensive historical data, which is crucial for solving scientific and engineering challenges. Traditional transformer-based methods process historical segments individually, leading to a limited view that overlooks distant dependencies within the entire time series. In this paper, we introduce the Segment-Recurrent Transformer (SRTrans), designed to provide a more comprehensive understanding of historical time series dynamics. By incorporating segment-level recurrence into the Transformer, our model enhances inter-segment information flow, capturing longer-term and global dependencies. We also propose a multi-scale adaptive fusion module that efficiently integrates diverse patterns using a variable-scale chunking mechanism and a weight-mixing strategy. Additionally, our spectrum purge operation improves data preprocessing by extracting significant long-term patterns from the frequency domain. Extensive experiments on eight real-world datasets demonstrate SRTrans’s effectiveness in accuracy and efficiency, offering a promising new solution for LTSF tasks.

Extracting a small subset of crucial rationales from the full input is a key problem in explainability research. The most widely used fundamental criterion for rationale extraction is the maximum mutual information (MMI) criterion. In this paper, we first demonstrate that MMI suffers from diminishing marginal returns. Once part of the rationale has been identified, finding the remaining portions contributes only marginally to increasing the mutual information, making it difficult to use MMI to locate the rest. In contrast to MMI that aims to reproduce the prediction, we seek to identify the parts of the input that the network can actually utilize. This is achieved by comparing how different rationale candidates match the capability space of the weight matrix. The weight matrix of a neural network is typically low-rank, meaning that the linear combinations of its column vectors can only cover part of the directions in a high-dimensional space (high-dimension: the dimensions of an input vector). If an input is fully utilized by the network, it generally matches these directions (e.g., a portion of a hypersphere), resulting in a representation with a high norm. Conversely, if an input primarily falls outside (orthogonal to) these directions, its representation norm will approach zero, behaving like noise that the network cannot effectively utilize. Building on this, we propose using the norms of rationale candidates as an alternative objective to MMI. Through experiments on four text classification datasets and one graph classification dataset using three network architectures (GRUs, BERT, and GCN), we show that our method outperforms MMI and its improved variants in identifying better rationales. We also compare our method with a representative LLM (llama-3.1-8b-instruct) and find that our simple method gets comparable results to it and can sometimes even outperform it.

A major challenge in Federated Learning (FL) is that the data is heterogeneous in terms of features among clients, which significantly degrades the performance of the global model due to its incurred client drift. To solve this problem, some works propose leveraging the global knowledge, e.g., class prototypes, to regularize the output of the local model to mitigate the local model drift. Despite their effectiveness, the input data of the local model are still heterogeneous, causing the parameters to not fully align. Therefore, to fill this gap, this paper proposes a novel method named FedRF which aligns local models on the input side by reusing the intermediate features to approach the global ones. Specifically, we use the intermediate features output by the global model on local data as input to assist in training the local model. Experimental results demonstrate that the proposed method significantly improves the performance of the global model as compared to baselines.

Low-rank adaptation (LoRA) is a widely used parameter-efficient fine-tuning method. In standard LoRA layers, one of the matrices, $A$ or $B$, is initialized to zero, ensuring that fine-tuning starts from the pretrained model. However, there is no theoretical support for this practice. In this paper, we investigate the impact of non-zero initialization on LoRA’s fine-tuning dynamics from an infinite-width perspective. Our analysis reveals that, compared to zero initialization, simultaneously initializing $A$ and $B$ to non-zero values improves LoRA’s robustness to suboptimal learning rates, particularly smaller ones. Further analysis indicates that although the non-zero initialization of $AB$ introduces random noise into the pretrained weight, it generally does not affect fine-tuning performance. In other words, fine-tuning does not need to strictly start from the pretrained model. The validity of our findings is confirmed through extensive experiments across various models and datasets. The code is available at https://github.com/Leopold1423/non_zero_lora-icml25.

To improve the training efficiency of federated learning (FL), previous research has employed low-rank decomposition techniques to reduce communication overhead. In this paper, we seek to enhance the performance of these low-rank decomposition methods. Specifically, we focus on three key issues related to decomposition in FL: what to decompose, how to decompose, and how to aggregate. Subsequently, we introduce three novel techniques: Model Update Decomposition (MUD), Block-wise Kronecker Decomposition (BKD), and Aggregation-Aware Decomposition (AAD), each targeting a specific issue. These techniques are complementary and can be applied simultaneously to achieve optimal performance. Additionally, we provide a rigorous theoretical analysis to ensure the convergence of the proposed MUD. Extensive experimental results show that our approach achieves faster convergence and superior accuracy compared to relevant baseline methods. The code is available at https://github.com/Leopold1423/fedmud-icml25.

Continual Federated Learning (CFL) allows distributed devices to collaboratively learn novel concepts from continuously shifting training data while avoiding \textit{knowledge forgetting} of previously seen tasks. To tackle this challenge, most current CFL approaches rely on extensive rehearsal of previous data. Despite effectiveness, rehearsal comes at a cost to memory, and it may also violate data privacy. Considering these, we seek to apply regularization techniques to CFL by considering their cost-efficient properties that do not require sample caching or rehearsal. Specifically, we first apply traditional regularization techniques to CFL and observe that existing regularization techniques, especially synaptic intelligence, can achieve promising results under homogeneous data distribution but fail when the data is heterogeneous. Based on this observation, we propose a simple yet effective regularization algorithm for CFL named \textbf{FedSSI}, which tailors the synaptic intelligence for the CFL with heterogeneous data settings. FedSSI can not only reduce computational overhead without rehearsal but also address the data heterogeneity issue. Extensive experiments show that FedSSI achieves superior performance compared to state-of-the-art methods.

This study investigates the self-rationalization framework constructed with a cooperative game, where a generator initially extracts the most informative segment from raw input, and a subsequent predictor utilizes the selected subset for its input. The generator and predictor are trained collaboratively to maximize prediction accuracy. In this paper, we first uncover a potential caveat: such a cooperative game could unintentionally introduce a sampling bias during rationale extraction. Specifically, the generator might inadvertently create an incorrect correlation between the selected rationale candidate and the label, even when they are semantically unrelated in the original dataset. Subsequently, we elucidate the origins of this bias using both detailed theoretical analysis and empirical evidence. Our findings suggest a direction for inspecting these correlations through attacks, based on which we further introduce an instruction to prevent the predictor from learning the correlations. Through experiments on six text classification datasets and two graph classification datasets using three network architectures (GRUs, BERT, and GCN), we show that our method significantly outperforms recent rationalization methods.

Cross-domain few-shot segmentation (CD-FSS) is proposed to first pre-train the model on a source-domain dataset with sufficient samples, and then transfer the model to target-domain datasets where only a few training samples are available for efficient finetuning. There are majorly two challenges in this task: (1) the domain gap and (2) finetuning with scarce data. To solve these challenges, we revisit the adapter-based methods, and discover an intriguing insight not explored in previous works: the adapter not only helps the fine-tuning of downstream tasks but also naturally serves as a domain information decoupler. Then, we delve into this finding for an interpretation, and we find the model's inherent structure could lead to a natural decoupling of domain information. Building upon this insight, we propose the Domain Feature Navigator (DFN), which is a structure-based decoupler instead of loss-based ones like current works, to capture domain-specific information, thereby directing the model's attention towards domain-agnostic knowledge. Moreover, to prevent the potential excessive overfitting of DFN during the source-domain training, we further design the SAM-SVN method to constrain DFN from learning sample-specific knowledge. On target domains, we freeze the model and fine-tune the DFN to learn knowledge specific to target domains. Extensive experiments demonstrate that our method surpasses the state-of-the-art method in CD-FSS significantly by 2.69% and 4.68% average MIoU in 1-shot and 5-shot scenarios, respectively.

Cross-Domain Few-Shot Segmentation (CD-FSS) aims to transfer knowledge from a large-scale source-domain dataset to unseen target-domain datasets with limited annotated samples. Current methods typically compare the distance between training and testing samples for mask prediction. However, a problem of feature entanglement exists in this well-adopted method, which binds multiple patterns together and harms the transferability. However, we find an entanglement problem exists in this widely adopted method, which tends to bind source-domain patterns together and make each of them hard to transfer. In this paper, we aim to address this problem for the CD-FSS task. We first find a natural decomposition of the ViT structure, based on which we delve into the entanglement problem for an interpretation. We find the decomposed ViT components are crossly compared between images in distance calculation, where the rational comparisons are entangled with those meaningless ones by their equal importance, leading to the entanglement problem. Based on this interpretation, we further propose to address the entanglement problem by learning to weigh for all comparisons of ViT components, which learn disentangled features and re-compose them for the CD-FSS task, benefiting both the generalization and finetuning. Experiments show that our model outperforms the state-of-the-art CD-FSS method by 1.92% and 1.88% in average accuracy under 1-shot and 5-shot settings, respectively.

Semi-supervised Federated Learning (SSFL) is a promising approach that allows clients to collaboratively train a global model in the absence of their local data labels. The key step of SSFL is the re-labeling where each client adopts two types of available models, namely global and local models, to re-label the local data. While various technologies such as using the global model or the average of two models have been proposed to conduct the re-labeling step, little literature delves deeply into the performance dominance and limitations of the two models. In this paper, we first theoretically and empirically demonstrate that the local model achieves higher re-labeling accuracy over local data while the global model can progressively improve the re-labeling performance by introducing the extra data knowledge of other clients. Based on these findings, we propose BSemiFL which re-labels the local data through the collaboration between the local and global model in a Bayesian approach. Specifically, to re-label any given local sample, BSemiFL first uses Bayesian inference to assess the closeness of the local/global model to the sample. Then, it applies a weighted combination of their pseudo labels, using the closeness as the weights. Theoretical analysis shows that the labeling error of our method is smaller than that of simply using the global model, the local model, or their simple average. Experimental results show that BSemiFL improves the performance by up to $9.8\%$ as compared to state-of-the-art methods.

Vision Transformer (ViT) has achieved remarkable success due to its large-scale pretraining on general domains, but it still faces challenges when applying it to downstream distant domains that have only scarce training data, which gives rise to the Cross-Domain Few-Shot Learning (CDFSL) task. Inspired by Self-Attention's insensitivity to token orders, we find an interesting phenomenon neglected in current works: disrupting the continuity of image tokens (i.e., making pixels not smoothly transited across patches) in ViT leads to a noticeable performance decline in the general (source) domain but only a marginal decrease in downstream target domains. This questions the role of image tokens' continuity in ViT's generalization under large domain gaps. In this paper, we delve into this phenomenon for an interpretation. We find continuity aids ViT in learning larger spatial patterns, which are harder to transfer than smaller ones, enlarging domain distances. Meanwhile, it implies that only smaller patterns within each patch could be transferred under extreme domain gaps. Based on this interpretation, we further propose a simple yet effective method for CDFSL that better disrupts the continuity of image tokens, encouraging the model to rely less on large patterns and more on smaller ones. Extensive experiments show the effectiveness of our method in reducing domain gaps and outperforming state-of-the-art works. Codes and models are available at https://github.com/shuaiyi308/ReCIT.

Cross-domain few-shot learning (CDFSL) aims to transfer knowledge from a data-sufficient source domain to data-scarce target domains. Although Vision Transformer (ViT) has shown superior capability in many vision tasks, its transferability against huge domain gaps in CDFSL is still under-explored. In this paper, we find an intriguing phenomenon: during the source-domain training, prompt tuning, as a common way to train ViT, could be harmful for the generalization of ViT in target domains, but setting them to random noises (i.e., random registers) could consistently improve target-domain performance. We then delve into this phenomenon for an interpretation. We find that learnable prompts capture domain information during the training on the source dataset, which views irrelevant visual patterns as vital cues for recognition. This can be viewed as a kind of overfitting and increases the sharpness of the loss landscapes. In contrast, random registers are essentially a novel way of perturbing attention for the sharpness-aware minimization, which helps the model find a flattened minimum in loss landscapes, increasing the transferability. Based on this phenomenon and interpretation, we further propose a simple but effective approach for CDFSL to enhance the perturbation on attention maps by adding random registers on the semantic regions of image tokens, improving the effectiveness and efficiency of random registers. Extensive experiments on four benchmarks validate our rationale and state-of-the-art performance. Codes and models are available at https://github.com/shuaiyi308/REAP.

Missing value imputation is a critical data pre-processing technique for machine learning tasks. However, the imputation accuracy would degrade significantly when the volume of known values is limited. To address this issue, this paper proposes a Federated Transformed Sampling Comparison (FTSC) method for imputing missing values, which is based on the optimal transport divergence of two samplings in latent representations space. In detail, assuming that the data samplings come from the same distribution, an invertible neural network (INN) is designed to transform the original sampling data into latent representations within each client. Then, based on the geodesic triangle inequality of the Wasserstein distance, the Wasserstein distance of these latent representations is estimated federatively and modeled as the loss function to jointly optimize the imputation values and the INN parameters. We also theoretically prove the effectiveness of Wasserstein distance estimation and INN training methods. Extensive experiments are implemented on both real and synthetic datasets, and experiment results show that our method in all three missing mechanisms is stable and can outperform several effective imputation methods.

Despite Federated Learning (FL) employing gradient aggregation at the server for distributed training to prevent the privacy leakage of raw data, private information can still be divulged through the analysis of uploaded gradients from clients. Substantial efforts have been made to integrate local differential privacy (LDP) into the system to achieve a strict privacy guarantee. However, existing methods fail to take practical issues into account by merely perturbing each sample with the same mechanism while each client may have their own privacy preferences on privacy-sensitive information (PSI), which is not uniformly distributed across the raw data. In such a case, excessive privacy protection from private-insensitive information can additionally introduce unnecessary noise, which may degrade the model performance. In this work, we study the PSI within data and develop FedRE, that can simultaneously achieve robustness and effectiveness benefits with LDP protection. More specifically, we first define PSI with regard to the privacy preferences of each client. Then, we optimize the LDP by allocating less privacy budget to gradients with higher PSI in a layer-wise manner, thus providing a stricter privacy guarantee for PSI. Furthermore, to mitigate the performance degradation caused by LDP, we design a parameter aggregation mechanism based on the distribution of the perturbed information. We conducted experiments with text tamper detection on T-SROIE and DocTamper datasets, and FedRE achieves competitive performance compared to state-of-the-art methods.

Low-rank adaptation (LoRA) is a parameter-efficient fine-tuning (PEFT) method widely used in large language models (LLMs). LoRA essentially describes the projection of an input space into a low-dimensional output space, with the dimensionality determined by the LoRA rank. In standard LoRA, all input tokens share the same weights and undergo an identical input-output projection. This limits LoRA's ability to capture token-specific information due to the inherent semantic differences among tokens. To address this limitation, we propose **Token-wise Projected Low-Rank Adaptation (TopLoRA)**, which dynamically adjusts LoRA weights according to the input token, thereby learning token-wise input-output projections in an end-to-end manner. Formally, the weights of TopLoRA can be expressed as $B\Sigma_X A$, where $A$ and $B$ are low-rank matrices (as in standard LoRA), and $\Sigma_X$ is a diagonal matrix generated from each input token $X$. Notably, TopLoRA does not increase the rank of LoRA weights but achieves more granular adaptation by learning token-wise LoRA weights (i.e., token-wise input-output projections). Extensive experiments across multiple models and datasets demonstrate that TopLoRA consistently outperforms LoRA and its variants. The code is available at https://github.com/Leopold1423/toplora-neurips25.

The current Federated Recommendation System (FedRS) focuses on personalized recommendation services and assumes clients are personalized IoT devices (e.g., Mobile phones). In this paper, we deeply dive into new but practical FedRS applications within the joint venture ecosystem. Subsidiaries engage as participants with their users and items. However, in such a situation, merely exchanging item embedding is insufficient, as user bases always exhibit both overlaps and exclusive segments, demonstrating the complexity of user information. Meanwhile, directly uploading user information is a violation of privacy and unacceptable. To tackle the above challenges, we propose an efficient and privacy-enhanced federated recommendation for the joint venture ecosystem (FR-JVE) that each client transfers more common knowledge from other clients with a distilled user's \textit{rating preference} from the local dataset. More specifically, we first transform the local data into a new format and apply model inversion techniques to distill the rating preference with frozen user gradients before the federated training. Then, a bridge function is employed on each client side to align the local rating preference and aggregated global preference in a privacy-friendly manner. Finally, each client matches similar users to make a better prediction for overlapped users. From a theoretical perspective, we analyze how effectively FR-JVE can guarantee user privacy. Empirically, we show that FR-JVE achieves superior performance compared to state-of-the-art methods.

Federated Continual Learning (FCL) aims to enable sequential privacy-preserving model training on streams of incoming data that vary in edge devices by preserving previous knowledge while adapting to new data. Current FCL literature focuses on restricted data privacy and access to previously seen data while imposing no constraints on the training overhead. This is unreasonable for FCL applications in real-world scenarios, where edge devices are primarily constrained by resources such as storage, computational budget, and label rate. We revisit this problem with a large-scale benchmark and analyze the performance of state-of-the-art FCL approaches under different resource-constrained settings. Various typical FCL techniques and six datasets in two incremental learning scenarios (Class-IL and Domain-IL) are involved in our experiments. Through extensive experiments amounting to a total of over 1,000+ GPU hours, we find that, under limited resource-constrained settings, existing FCL approaches, with no exception, fail to achieve the expected performance. Our conclusions are consistent in the sensitivity analysis. This suggests that most existing FCL methods are particularly too resource-dependent for real-world deployment. Moreover, we study the performance of typical FCL techniques with resource constraints and shed light on future research directions in FCL.

Model-Heterogeneous Federated Learning (Hetero-FL) has attracted growing attention for its ability to aggregate knowledge from heterogeneous models while keeping private data locally. To better aggregate knowledge from clients, ensemble distillation, as a widely used and effective technique, is often employed after global aggregation to enhance the performance of the global model. However, simply combining Hetero-FL and ensemble distillation does not always yield promising results and can make the training process unstable. The reason is that existing methods primarily focus on logit distillation, which, while being model-agnostic with softmax predictions, fails to compensate for the knowledge bias arising from heterogeneous models. To tackle this challenge, we propose a stable and efficient Feature Distillation for model-heterogeneous Federated learning, dubbed FedFD, that can incorporate aligned feature information via orthogonal projection to integrate knowledge from heterogeneous models better. Specifically, a new feature-based ensemble federated knowledge distillation paradigm is proposed. The global model on the server needs to maintain a projection layer for each client-side model architecture to align the features separately. Orthogonal techniques are employed to re-parameterize the projection layer to mitigate knowledge bias from heterogeneous models and thus maximize the distilled knowledge. Extensive experiments show that FedFD achieves superior performance compared to state-of-the-art methods.

Out-of-distribution generalization under distributional shifts remains a critical challenge for graph neural networks. Existing methods generally adopt the Invariant Risk Minimization (IRM) framework, requiring costly environment annotations or heuristically generated synthetic splits. To circumvent these limitations, in this work, we aim to develop an IRM-free method for capturing causal subgraphs. We first identify that causal subgraphs exhibit substantially smaller distributional variations than non-causal components across diverse environments, which we formalize as the Invariant Distribution Criterion and theoretically prove in this paper. Building on this criterion, we systematically uncover the quantitative relationship between distributional shift and representation norm for identifying the causal subgraph, and investigate its underlying mechanisms in depth. Finally, we propose an IRM-free method by introducing a norm-guided invariant distribution objective for causal subgraph discovery and prediction. Extensive experiments on two widely used benchmarks demonstrate that our method consistently outperforms state-of-the-art methods in graph generalization. Code is available at https://github.com/anders1123/IDG.

Large vision-language models (LVLMs) excel at multimodal understanding but suffer from high computational costs due to redundant vision tokens. Existing pruning methods typically rely on single-layer attention scores to rank and prune redundant visual tokens to solve this inefficiency. However, as the interaction between tokens and layers is complicated, this raises a basic question: Is such a simple single-layer criterion sufficient to identify redundancy? To answer this question, we rethink the emergence of redundant visual tokens from a fundamental perspective: information flow, which models the interaction between tokens and layers by capturing how information moves between tokens across layers. We find (1) the CLS token acts as an information relay, which can simplify the complicated flow analysis; (2) the redundancy emerges progressively and dynamically via layer-wise attention concentration; and (3) relying solely on attention scores from single layers can lead to contradictory redundancy identification. Based on this, we propose FlowCut, an information-flow-aware pruning framework, mitigating the insufficiency of the current criterion for identifying redundant tokens and better aligning with the model's inherent behaviors. Extensive experiments show FlowCut achieves superior results, outperforming SoTA by 1.6% on LLaVA-1.5-7B with 88.9% token reduction, and by 4.3% on LLaVA-NeXT-7B with 94.4% reduction, delivering 3.2$\times$ speed-up in the prefilling stage. Our code is available at https://github.com/TungChintao/FlowCut.

Multimodal federated learning (MMFL) has gained increasing popularity due to its ability to leverage the correlation between various modalities, meanwhile preserving data privacy for different clients. However, recent studies show that correlation between modalities increase the vulnerability of federated learning against Gradient Inversion Attack (GIA). The complicated situation of MMFL privacy preserving can be summarized as follows: 1) different modality transmits different amounts of information, thus requires various protection strength; 2) correlation between modalities should be taken into account. This paper introduces an information theory perspective to analyze the leaked privacy in process of MMFL, and tries to propose a more reasonable protection method \textbf{Sec-MMFL} based on assessing different information leakage possibilities of each modality by conditional mutual information and adjust the corresponding protection strength. Moreover, we use mutual information to reduce the cross-modality information leakage in MMFL. Experiments have proven that our method can bring more balanced and comprehensive protection at an acceptable cost.

With the increasing sophistication of Large Language Models (LLMs), it is crucial to develop reliable methods to accurately identify whether an interlocutor in real-time dialogue is human or chatbot. However, existing detection methods are primarily designed for analyzing full documents, not the unique dynamics and characteristics of dialogue. These approaches frequently overlook the nuances of interaction that are essential in conversational contexts. This work identifies two key patterns in dialogues: (1) Human-Human (H-H) interactions exhibit significant bidirectional sentiment influence, while (2) Human-Chatbot (H-C) interactions display a clear asymmetric pattern. We propose an innovative approach named ChatbotID, which applies the Granger Causality Test (GCT) to extract a novel set of interactional features that capture the evolving, predictive relationships between conversational attributes. By synergistically fusing these GCT-based interactional features with contextual embeddings, and optimizing the model through a meticulous loss function. Experimental results across multiple datasets and detection models demonstrate the effectiveness of our framework, with significant improvements in accuracy for distinguishing between H-H and H-C dialogues.

Federated Recommendation System (FRS) usually offers recommendation services for users while keeping their data locally to ensure privacy. Currently, most FRS literature assumes that fixed users participate in federated training with personal IoT devices (e.g., mobile phones and PC). However, users may join incrementally, and retraining the entire FRS for each new participating user is unfeasible due to the high training costs and the limited global knowledge contribution from a small number of new users. To guarantee the quality service for these new users, we take a dive into the federated recommendation for cold-start users, a novel scenario where the new participating users can directly obtain a promising recommendation without comprehensive training with all participating users by leveraging both transferred knowledge from the converged warm clients and the knowledge learned from the local data.

Privacy-preserving Cross-Domain Recommendation (CDR) has been extensively studied to address the cold-start problem using auxiliary source domains while simultaneously protecting sensitive information. However, existing privacy-preserving CDR methods rely heavily on transferring sensitive user embeddings or behaviour logs, which leads to adopt privacy methods to distort the data patterns before transferring it to the target domain. The distorted information can compromise overall performance during the knowledge transfer process. To overcome these challenges, our approach differs from existing privacy-preserving methods that focus on safeguarding user-sensitive information. Instead, we concentrate on distilling transferable knowledge from insensitive item embeddings, which we refer to as prototypes. Specifically, we propose a conditional model inversion mechanism to accurately distill prototypes for individual users. We have designed a new data format and corresponding learning paradigm for distilling transferable prototypes from traditional recommendation models using model inversion. These prototypes facilitate bridging the domain shift between distinct source and target domains in a privacy-friendly manner. Additionally, they enable the identification of top-k users in the target domain to substitute for cold-start users prediction. We conduct extensive experiments across large real-world datasets, and the results substantiate the effectiveness of PFCDR https://github.com/walcheng/PFCDR.

Federated learning (FL) has emerged as a significant distributed machine learning paradigm. It allows the training of a global model through user collaboration without the necessity of sharing their original data. Traditional FL generally assumes that each client's data remains fixed or static. However, in real-world scenarios, data typically arrives incrementally, leading to a dynamically expanding data domain. In this study, we examine catastrophic forgetting within Federated Incremental Learning (FIL) and focus on the training resources, where edge clients may not have sufficient storage to keep all data or computational budget to implement complex algorithms designed for the server-based environment. We propose a general and low-cost framework for FIL named Re-Fed+, which is designed to help clients cache important samples for replay. Specifically, when a new task arrives, each client initially caches selected previous samples based on their global and local significance. The client then trains the local model using both the cached samples and the new task samples. From a theoretical perspective, we analyze how effectively Re-Fed+ can identify significant samples for replay to alleviate the catastrophic forgetting issue. Empirically, we show that Re-Fed+ achieves competitive performance compared to state-of-the-art methods.

Abstract Rationalization is a framework that aims to build self-explanatory NLP models by extracting a subset of human-intelligible pieces of their inputting texts. It involves a cooperative game where a selector selects the most human-intelligible parts of the input as the rationale, followed by a predictor that makes predictions based on these selected rationales. Existing literature uses the cross-entropy between the model’s predictions and the ground-truth labels to measure the informativeness of the selected rationales, guiding the selector to choose better ones. In this study, we first theoretically analyze the objective of rationalization by decomposing it into two parts: the model-agnostic informativeness of the rationale candidates and the predictor’s degree of fit. We then provide various empirical evidence to support that, under this framework, the selector tends to sample from a limited small region, causing the predictor to overfit these localized areas. This results in a significant mismatch between the cross-entropy objective and the informativeness of the rationale candidates, leading to suboptimal solutions. To address this issue, we propose a simple yet effective method that introduces random vicinal1 perturbations to the selected rationale candidates. This approach broadens the predictor’s assessment to a vicinity around the selected rationale candidate. Compared to recent competitive methods, our method significantly improves rationale quality (by up to 6.6%) across six widely used classification datasets. The term “vicinal” is borrowed from vicinal risk minimization (Chapelle et al., 2000); “vicinal” means neighboring or adjacent.

Post-deduplication in traditional cloud environments primarily focuses on single-node, where delta compression is performed on the same deduplication node located on server side. However, with data explosion, the multi-node post-deduplication, also called global deduplication, has become a hot issue in research communities, which aims to simultaneously execute delta compression on data distributed across all nodes. Simply setting up single-node deduplication systems on multi-node environments would significantly affect storage utilization and incur secondary overhead from file migration. Nevertheless, existing global deduplication solutions suffer from lower data compression ratios and high computational overhead due to their resemblance detection's inherent limitations and overly coarse granularities. Similar blocks typically have high correlations between sub-blocks; inspired by this observation, we propose IBNR (Intra-Block Neighborhood Relationship-Based Resemblance Detection for High-Performance Multi-Node Post-Deduplication), which introduces a novel resemblance detection based on relationships between sub-blocks and determines the ownership of blocks in entry stage to achieve efficient global deduplication. Furthermore, the by-products of IBNR have shown powerful scalability by replacing internal resemblance detection scheme with existing solutions on practical workloads. Experimental results indicate that IBNR outperforms state-of-the-art solutions, achieving an average 1.99× data reduction ratio and varying degrees of improvement across other key metrics.

Detecting similar data is crucial for optimizing file storage and transmission in HTTP protocols and Content Delivery Networks. Traditional MinHash methods encounter significant efficiency challenges due to their reliance on K-shingle structures, resulting in high computational costs and storage requirements. Additionally, these methods expose privacy risks in cloud environments, where sensitive information can be inferred from MinHash signatures. To address both efficiency and security concerns, we propose Horse-MinHash, which integrates a fast, content-defined feature extraction scheme with a non-interactive zero-knowledge proof-based similarity estimation method. Our approach significantly enhances computational efficiency while ensuring robust privacy protection by preventing plaintext exposure. Experimental results demonstrate that Horse-MinHash achieves lower mean squared error in Jaccard similarity estimation and reduces time overhead for average block sizes of 16 KB or more, outperforming state-of-the-art methods.

The field of graph foundation models (GFMs) has seen a dramatic rise in interest in recent years. Their powerful generalization ability is believed to be endowed by self-supervised pre-training and downstream tuning techniques. There is a wide variety of knowledge patterns embedded in the graph data, such as node properties and clusters, which are crucial for learning generalized representations for GFMs. We present a comprehensive survey of self-supervised GFMs from a novel knowledge-based perspective. Our main contribution is a knowledge-based taxonomy that categorizes self-supervised graph models by the specific graph knowledge utilized: microscopic (nodes, links, etc.), mesoscopic (context, clusters, etc.), and macroscopic (global structure, manifolds, etc.). It covers a total of 9 knowledge categories and 300 references for self-supervised pre-training as well as various downstream tuning strategies. Such a knowledge-based taxonomy allows us to more clearly re-examine potential GFM architectures, including large language models (LLMs), as well as provide deeper insights for constructing future GFMs.

In recommender systems, the patterns of user behaviors (e.g., purchase, click) may vary greatly in different contexts (e.g., time and location). This is because user behavior is jointly determined by two types of factors: intrinsic factors , which reflect consistent user preference, and extrinsic factors , which reflect external incentives that may vary in different contexts. Differentiating between intrinsic and extrinsic factors helps learn user behaviors better. However, existing studies have only considered differentiating them from a single, pre-defined context (e.g., time or location), ignoring the fact that a user’s extrinsic factors may be influenced by the interplay of various contexts at the same time. In this article, we propose the intrinsic-extrinsic disentangled recommendation (IEDR) model, a generic framework that differentiates intrinsic from extrinsic factors considering various contexts simultaneously, enabling more accurate differentiation of factors and hence the improvement of recommendation accuracy. IEDR contains a context-invariant contrastive learning component to capture intrinsic factors, and a disentanglement component to extract extrinsic factors under the interplay of various contexts. The two components work together to achieve effective factor learning. Extensive experiments on real-world datasets demonstrate IEDR’s effectiveness in learning disentangled factors and significantly improving recommendation accuracy by up to 4% in NDCG.

Federated learning (FL) is an emerging distributed machine learning paradigm that enables collaborative model training without sharing local data. Despite its advantages, FL suffers from substantial communication overhead, which can affect training efficiency. Recent efforts have mitigated this issue by quantizing model updates to reduce communication costs. However, most existing methods apply quantization only after local training, introducing quantization errors into the trained parameters and potentially degrading model accuracy. In this paper, we propose Federated Bit Freezing (FedBiF), a novel FL framework that directly learns quantized model parameters during local training. In each communication round, the server first quantizes the model parameters and transmits them to the clients. FedBiF then allows each client to update only a single bit of the multi-bit parameter representation, freezing the remaining bits. This bit-by-bit update strategy reduces each parameter update to one bit while maintaining high precision in parameter representation. Extensive experiments are conducted on five widely used datasets under both IID and Non-IID settings. The results demonstrate that FedBiF not only achieves superior communication compression but also promotes sparsity in the resulting models. Notably, FedBiF attains accuracy comparable to FedAvg, even when using only 1 bit-per-parameter (bpp) for uplink and 3 bpp for downlink communication. The code is available athttps://github.com/Leopold1423/fedbif-tpds25.

Few-shot intent classification and slot filling are important but challenging tasks due to the scarcity of finely labeled data. Therefore, current works first train a model on source domains with sufficiently labeled data, and then transfer the model to target domains where only rarely labeled data is available. However, experience transferring as a whole usually suffers from gaps that exist among source domains and target domains. For instance, transferring domain-specific-knowledge-related experience is difficult. To tackle this problem, we propose a new method that explicitly decouples the transferring of general-semantic-representation-related experience and the domain-specific-knowledge-related experience. Specifically, for domain-specific-knowledge-related experience, we design two modules to capture intent-slot relation and slot-slot relation respectively. Extensive experiments on Snips and FewJoint datasets show that our method achieves state-of-the-art performance. The method improves the joint accuracy metric from 27.72% to 42.20% in the 1-shot setting, and from 46.54% to 60.79% in the 5-shot setting.

In Federated Learning (FL), the data in each client is typically assumed fixed or static. However, data often comes in an incremental manner in real-world applications, where the data domain may increase dynamically. In this work, we study catastrophic forgetting with data heterogeneity in Federated Incremental Learning (FIL) scenarios where edge clients may lack enough storage space to retain full data. We propose to employ a simple, generic frame-work for FIL named Re-Fed, which can coordinate each client to cache important samples for replay. More specifically, when a new task arrives, each client first caches selected previous samples based on their global and local im-portance. Then, the client trains the local model with both the cached samples and the samples from the new task. The-oretically, we analyze the ability of Re-Fed to discover important samples for replay thus alleviating the catastrophic forgetting problem. Moreover, we empirically show that Re-Fed achieves competitive performance compared to state-of-the-art methods.

Cross-domain few-shot learning (CDFSL) aims to acquire knowledge from limited training data in the target domain by leveraging prior knowledge transferred from source domains with abundant training samples. CDFSL faces challenges in transferring knowledge across dissimilar domains and fine-tuning models with limited training data. To address these challenges, we initially extend the analysis of loss landscapes from the parameter space to the representation space, which allows us to simultaneously interpret the transferring and fine-tuning difficulties of CDFSL models. We observe that sharp minima in the loss landscapes of the representation space result in representations that are hard to transfer and fine-tune. More-over, existing flatness-based methods have limited generalization ability due to their short-range flatness. To enhance the transferability and facilitate fine-tuning, we introduce a simple yet effective approach to achieve long-range flat-tening of the minima in the loss landscape. This approach considers representations that are differently normalized as minima in the loss landscape and flattens the high-loss region in the middle by randomly sampling interpolated representations. We implement this method as a new normalization layer that replaces the original one in both CNNs and ViTs. This layer is simple and lightweight, introducing only a minimal number of additional parameters. Experimental results on 8 datasets demonstrate that our approach outperforms state-of-the-art methods in terms of average accuracy. Moreover, our method achieves performance improvements of up to 9% compared to the current best approaches on individual datasets. Our code will be released.

Rationalization models, which select a subset of input text as rationale—crucial for humans to understand and trust predictions—have recently emerged as a prominent research area in eXplainable Artificial Intelligence (XAI). However, most of previous studies mainly focus on improving the quality of the rationale, ignoring its robustness to malicious attack. Specifically, whether the rationalization models can still generate high-quality rationale under the adversarial attack remains unknown. To explore this, this paper proposes UAT2E, which aims to undermine the explainability of rationalization models without altering their predictions, thereby eliciting distrust in these models from human users. UAT2E employs the gradient-based search on triggers and then inserts them into the original input to conduct both the non-target and target attack. Experimental results on five datasets reveal the vulnerability of rationalization models in terms of explanation, where they tend to select more meaningless tokens under attacks. Based on this, we make a series of recommendations for improving rationalization models in terms of explanation.

This article introduces PAGE, a parameterized generative interpretive framework. PAGE is capable of providing faithful explanations for any graph neural network without necessitating prior knowledge or internal details. Specifically, we train the autoencoder to generate explanatory substructures by designing appropriate training strategy. Due to the dimensionality reduction of features in the latent space of the autoencoder, it becomes easier to extract causal features leading to the model’s output, which can be easily employed to generate explanations. To accomplish this, we introduce an additional discriminator to capture the causality between latent causal features and the model’s output. By designing appropriate optimization objectives, the well-trained discriminator can be employed to constrain the encoder in generating enhanced causal features. Finally, these features are mapped to substructures of the input graph through the decoder to serve as explanations. Compared to existing methods, PAGE operates at the sample scale rather than nodes or edges, eliminating the need for perturbation or encoding processes as seen in previous methods. Experimental results on both artificially synthesized and real-world datasets demonstrate that our approach not only exhibits the highest faithfulness and accuracy but also significantly outperforms baseline models in terms of efficiency.

Transformer with self-attention was initially crafted to model language sequences, where discrete tokens (i.e., words) showcase high semantic density. However, when applied to time series token inputs (i.e., datapoints) with weak-density semantics and temporal redundancy, it faces challenges as these time-domain tokens impede its ability to capture the intricate latent properties of time series dynamics. While time-frequency transformation presents a viable solution by bringing forth a new space with heightened expressive power, existing approaches fall short of fully exploiting its potential. In response to these limitations, we propose a general-purpose transformer-based model, named Scattering Transformer, for multivariate time series forecasting and self-supervised representation learning. It is based on two innovative components: i) scattering self-attention mechanism incorporating wavelet key/value and standard query to unify the learning of cross-domain relationships between the time and wavelet domains; and ii) stochastic scaling positional encoding scheme that relies solely on order information, emulating longer sequence positions to generalize up to ultra-long horizon case. Extensive experiments on eight real-world benchmarks show the potential of our Scattering Transformer as a robust and versatile solution, showcasing its quadruple efficacy of non-stationary forecasting, ultra-long horizons forecasting, representation learning, and reduction in time and space complexity.

Deep learning models, usually trained on large datasets, have witnessed great achievements for both classification and regression tasks during past years. However, in practical settings, the datasets are commonly imbalanced where the number of data samples differs among labels (i.e.,categories or target values), leading to serious performance degradation of the trained models. Although many prior works have been proposed to solve the data imbalance problem for the classification task, there are still limited works considering the regression task. In this work, we identify the distinct challenges in regression problems compared to traditional classification problems, such as managing the continuous nature of the target variable and navigating fuzzy decision-making boundaries. To address these challenges, we propose an Adaptive Weighted Statistical Space Smoothing method (AWSSS), which alleviates the impact of imbalanced data by learning from continuously valued, imbalanced data and extending local similarity across the entire target range. AWSSS extracts statistical values from the original dataset and smoothes statistical features. Subsequently, it aligns features across different regions and applies adaptive weights to facilitate knowledge transfer between adjacent areas. Finally, AWSSS calculates the loss values, which are used to refine the model’s parameters through iterative updates. Extensive experiments conducted on various datasets demonstrate the effectiveness of the proposed method as compared to state-of-the-art methods.

Federated Learning (FL) has emerged as a foundational paradigm that enables collaborative training of deep neural networks across distributed clients while ensuring data privacy. However, most of the existing research primarily concentrates on supervised federated learning tailored for specific downstream tasks. In this paper, we concentrate on unsupervised federated learning, where clients collaborate to identify data patterns, structures, or representations. Contrastive learning methods have proven highly effective for unsupervised learning in server-based environments. However, a straightforward adaptation of the contrastive learning approach for this setting falls short. Upon analyzing client behaviors during training, we identified an unstable training process stemming from NonIID issues, which can result in diminished performance. To address this problem, we propose a novel Temperature-Adaption based unsupervised Federated prototype learning approach, termed FedTA. This method incorporates two straightforward yet potent solutions: 1. Prototype Enhancement and 2. Temperature-Adaption. Experimental results show that our proposed approach surpasses the state-of-the-art, achieving a performance accuracy improvement of up to 5.3%.

Rationalization empowers deep learning models with self-explaining capabilities through a cooperative game, where a generator selects a semantically consistent subset of the input as a rationale, and a subsequent predictor makes predictions based on the selected rationale. In this paper, we discover that rationalization is prone to a problem named rationale shift, which arises from the algorithmic bias of the cooperative game. Rationale shift refers to a situation where the semantics of the selected rationale may deviate from the original input, but the predictor still produces accurate predictions based on the deviation, resulting in a compromised generator with misleading feedback. To address this issue, we first demonstrate the importance of the alignment between the rationale and the full input through both empirical observations and theoretical analysis. Subsequently, we introduce a novel approach called DAR (Discriminatively Aligned Rationalization), which utilizes an auxiliary module pretrained on the full input to discriminatively align the selected rationale and the original input. We theoretically illustrate how DAR accomplishes the desired alignment, thereby overcoming the rationale shift problem. The experiments on two widely used real-world benchmarks show that the proposed method significantly improves the explanation quality (measured by the overlap between the model-selected explanation and the human-annotated rationale) as compared to state-of-the-art techniques. Additionally, results on two synthetic settings further validate the effectiveness of DAR in addressing the rationale shift problem.

In Federated Learning (FL), model aggregation is pivotal. It involves a global server iteratively aggregating client local trained models in successive rounds without accessing private data. Traditional methods typically aggregate the local models from the current round alone. However, due to the statistical heterogeneity across clients, the local models from different clients may be greatly diverse, making the obtained global model incapable of maintaining the specific knowledge of each local model. In this paper, we introduce a novel method, FedCDA, which selectively aggregates cross-round local models, decreasing discrepancies between the global model and local models. The principle behind FedCDA is that due to the different global model parameters received in different rounds and the non-convexity of deep neural networks, the local models from each client may converge to different local optima across rounds. Therefore, for each client, we select a local model from its several recent local models obtained in multiple rounds, where the local model is selected by minimizing its divergence from the local models of other clients. This ensures the aggregated global model remains close to all selected local models to maintain their data knowledge. Extensive experiments conducted on various models and datasets reveal our approach outperforms state-of-the-art aggregation methods.

Federated learning is a promising distributed machine learning paradigm that can effectively exploit large-scale data without exposing users’ privacy. However, it may incur significant communication overhead, thereby potentially impairing the training efficiency. To address this challenge, numerous studies suggest binarizing the model updates. Nonetheless, traditional methods usually binarize model updates in a post-training manner, resulting in significant approximation errors and consequent degradation in model accuracy. To this end, we propose Federated Binarization-Aware Training (FedBAT), a novel framework that directly learns binary model updates during the local training process, thus inherently reducing the approximation errors. FedBAT incorporates an innovative binarization operator, along with meticulously designed derivatives to facilitate efficient learning. In addition, we establish theoretical guarantees regarding the convergence of FedBAT. Extensive experiments are conducted on four popular datasets. The results show that FedBAT significantly accelerates the convergence and exceeds the accuracy of baselines by up to 9%, even surpassing that of FedAvg in some cases.

Few-shot class-incremental learning (FSCIL) is proposed to continually learn from novel classes with only a few samples after the (pre-)training on base classes with sufficient data. However, this remains a challenge. In contrast, humans can easily recognize novel classes with a few samples. Cognitive science demonstrates that an important component of such human capability is compositional learning. This involves identifying visual primitives from learned knowledge and then composing new concepts using these transferred primitives, making incremental learning both effective and interpretable. To imitate human compositional learning, we propose a cognitive-inspired method for the FSCIL task. We define and build a compositional model based on set similarities, and then equip it with a primitive composition module and a primitive reuse module. In the primitive composition module, we propose to utilize the Centered Kernel Alignment (CKA) similarity to approximate the similarity between primitive sets, allowing the training and evaluation based on primitive compositions. In the primitive reuse module, we enhance primitive reusability by classifying inputs based on primitives replaced with the closest primitives from other classes. Experiments on three datasets validate our method, showing it outperforms current state-of-the-art methods with improved interpretability. Our code is available at https://github.com/Zoilsen/Comp-FSCIL.

Federated Learning (FL) suffers from severe performance degradation due to the data heterogeneity among clients. Some existing work suggests that the fundamental reason is that data heterogeneity can cause local model drift, and therefore proposes to calibrate the direction of local updates to solve this problem. Though effective, existing methods generally take the model as a whole, which lacks a deep understanding of how the neurons within deep classification models evolve during local training to form model drift. In this paper, we bridge this gap by performing an intuitive and theoretical analysis of the activation changes of each neuron during local training. Our analysis shows that the high activation of some neurons on the samples of a certain class will be reduced during local training when these samples are not included in the client, which we call neuron drift, thus leading to the performance reduction of this class. Motivated by this, we propose a novel and simple algorithm called FedNLR, which utilizes Neuron-wise Learning Rates during the FL local training process. The principle behind this is to enhance the learning of neurons bound to local classes on local data knowledge while reducing the decay of non-local classes knowledge stored in neurons. Experimental results demonstrate that FedNLR achieves state-of-the-art performance on federated learning with popular deep neural networks.

Federated learning is a promising distributed training paradigm that effectively safeguards data privacy. However, it may involve significant communication costs, which hinders training efficiency. In this paper, we aim to enhance communication efficiency from a new perspective. Specifically, we request the distributed clients to find optimal model updates relative to global model parameters within predefined random noise. For this purpose, we propose Federated Masked Random Noise (FedMRN), a novel framework that enables clients to learn a 1-bit mask for each model parameter and apply masked random noise (i.e., the Hadamard product of random noise and masks) to represent model updates. To make FedMRN feasible, we propose an advanced mask training strategy, called progressive stochastic masking (PSM). After local training, each client only need to transmit local masks and a random seed to the server. Additionally, we provide theoretical guarantees for the convergence of FedMRN under both strongly convex and non-convex assumptions. Extensive experiments are conducted on four popular datasets. The results show that FedMRN exhibits superior convergence speed and test accuracy compared to relevant baselines, while attaining a similar level of accuracy as FedAvg.

Few-shot open-set recognition (FSOR) is a challenging task that requires a model to recognize known classes and identify unknown classes with limited labeled data. Existing approaches, particularly Negative-Prototype-Based methods, generate negative prototypes based solely on known class data. However, as the unknown space is infinite while the known space is limited, these methods suffer from limited representation capability. To address this limitation, we propose a novel approach, termed Diversified Negative Prototypes Generator (DNPG), which adopts the principle of "learning unknowns from unknowns." Our method leverages the unknown space information learned from base classes to generate more representative negative prototypes for novel classes. During the pre-training phase, we learn the unknown space representation of the base classes. This representation, along with inter-class relationships, is then utilized in the meta-learning process to construct negative prototypes for novel classes. To prevent prototype collapse and ensure adaptability to varying data compositions, we introduce the Swap Alignment (SA) module. Our DNPG model, by learning from the unknown space, generates negative prototypes that cover a broader unknown space, thereby achieving state-of-the-art performance on three standard FSOR datasets. The repository of this project is available at https://github.com/iCGY96/DNPG.

Humans exhibit a remarkable ability to learn quickly from a limited number of labeled samples, a capability that starkly contrasts with that of current machine learning systems. Unsupervised Few-Shot Learning (U-FSL) seeks to bridge this divide by reducing reliance on annotated datasets during initial training phases. In this work, we first quantitatively assess the impacts of Masked Image Modeling (MIM) and Contrastive Learning (CL) on few-shot learning tasks. Our findings highlight the respective limitations of MIM and CL in terms of discriminative and generalization abilities, which contribute to their underperformance in U-FSL contexts. To address these trade-offs between generalization and discriminability in unsupervised pretraining, we introduce a novel paradigm named Masked Image Contrastive Modeling (MICM). MICM creatively combines the targeted object learning strength of CL with the generalized visual feature learning capability of MIM, significantly enhancing its efficacy in downstream few-shot learning inference. Extensive experimental analyses confirm the advantages of MICM, demonstrating significant improvements in both generalization and discrimination capabilities for few-shot learning. Our comprehensive quantitative evaluations further substantiate the superiority of MICM, showing that our two-stage U-FSL framework based on MICM markedly outperforms existing leading baselines. The repository of this project is available at https://github.com/iCGY96/MICM.

Deep networks are known to be vulnerable to adversarial examples which are deliberately designed to mislead the trained model by introducing imperceptible perturbations to input samples. Compared to traditional perturbations crafted specifically for each data point, Universal Adversarial Perturbations (UAPs) are input-agnostic and shown to be more practical in the real world. However, UAPs are typically generated in a close-set scenario that shares the same classification task during the training and testing phases. This paper demonstrates the ineffectiveness of traditional UAPs in open-set scenarios like Few-Shot Learning (FSL). Through analysis, we identify two primary challenges that hinder the attacking process: the task shift and the semantic shift. To enhance the transferability of UAPs in FSL, we propose a unifying attacking framework addressing these two shifts. The task shift is addressed by aligning proxy tasks to the downstream tasks, while the semantic shift is handled by leveraging the generalizability of pre-trained encoders.The proposed Few-Shot Attacking FrameWork, denoted as FSAFW, can effectively generate UAPs across various FSL training paradigms and different downstream tasks. Our approach not only sets a new standard for state-of-the-art works but also significantly enhances attack performance, exceeding the baseline method by over 16\%.

An important line of research in the field of explainability is to extract a small subset of crucial rationales from the full input. The most widely used criterion for rationale extraction is the maximum mutual information (MMI) criterion. However, in certain datasets, there are spurious features non-causally correlated with the label and also get high mutual information, complicating the loss landscape of MMI. Although some penalty-based methods have been developed to penalize the spurious features (e.g., invariance penalty, intervention penalty, etc) to help MMI work better, these are merely remedial measures. In the optimization objectives of these methods, spurious features are still distinguished from plain noise, which hinders the discovery of causal rationales. This paper aims to develop a new criterion that treats spurious features as plain noise, allowing the model to work on datasets rich in spurious features as if it were working on clean datasets, thereby making rationale extraction easier. We theoretically observe that removing either plain noise or spurious features from the input does not alter the conditional distribution of the remaining components relative to the task label. However, significant changes in the conditional distribution occur only when causal features are eliminated. Based on this discovery, the paper proposes a criterion for \textbf{M}aximizing the \textbf{R}emaining \textbf{D}iscrepancy (MRD). Experiments on six widely used datasets show that our MRD criterion improves rationale quality (measured by the overlap with human-annotated rationales) by up to $10.4\%$ as compared to several recent competitive MMI variants. Code: \url{https://github.com/jugechengzi/Rationalization-MRD}.

Cross-domain few-shot segmentation (CD-FSS) is proposed to first pre-train the model on a large-scale source-domain dataset, and then transfer the model to data-scarce target-domain datasets for pixel-level segmentation. The significant domain gap between the source and target datasets leads to a sharp decline in the performance of existing few-shot segmentation (FSS) methods in cross-domain scenarios. In this work, we discover an intriguing phenomenon: simply filtering different frequency components for target domains can lead to a significant performance improvement, sometimes even as high as 14% mIoU. Then, we delve into this phenomenon for an interpretation, and find such improvements stem from the reduced inter-channel correlation in feature maps, which benefits CD-FSS with enhanced robustness against domain gaps and larger activated regions for segmentation. Based on this, we propose a lightweight frequency masker, which further reduces channel correlations by an Amplitude-Phase Masker (APM) module and an Adaptive Channel Phase Attention (ACPA) module. Notably, APM introduces only 0.01% additional parameters but improves the average performance by over 10%, and ACPA imports only 2.5% parameters but further improves the performance by over 1.5%, which significantly surpasses the state-of-the-art CD-FSS methods.

Cross-domain few-shot learning (CDFSL) is proposed to transfer knowledge from large-scale source-domain datasets to downstream target-domain datasets with only a few training samples. However, Vision Transformer (ViT), as a strong backbone network to achieve many top performances, is still under-explored in the CDFSL task in its transferability against large domain gaps. In this paper, we find an interesting phenomenon of ViT in the CDFSL task: by simply multiplying a temperature (even as small as 0) to the attention in ViT blocks, the target-domain performance consistently increases, even though the attention map is downgraded to a uniform map. In this paper, we delve into this phenomenon for an interpretation. Through experiments, we interpret this phenomenon as a remedy for the ineffective target-domain attention caused by the query-key attention mechanism under large domain gaps. Based on it, we further propose a simple but effective method for the CDFSL task to boost ViT's transferability by resisting the learning of query-key parameters and encouraging that of non-query-key ones. Experiments on four CDFSL datasets validate the rationale of our interpretation and method, showing we can consistently outperform state-of-the-art methods. Our codes are available at https://github.com/Zoilsen/Attn_Temp_CDFSL.

Vision Transformer (ViT) has shown great power in learning from large-scale datasets. However, collecting sufficient data for expert knowledge is always difficult. To handle this problem, Cross-Domain Few-Shot Learning (CDFSL) has been proposed to transfer the source-domain knowledge learned from sufficient data to target domains where only scarce data is available. In this paper, we find an intriguing phenomenon neglected by previous works for the CDFSL task based on ViT: leaving the CLS token to random initialization, instead of loading source-domain trained parameters, could consistently improve target-domain performance. We then delve into this phenomenon for an interpretation. We find **the CLS token naturally absorbs domain information** due to the inherent structure of the ViT, which is represented as the low-frequency component in the Fourier frequency space of images. Based on this phenomenon and interpretation, we further propose a method for the CDFSL task to decouple the domain information in the CLS token during the source-domain training, and adapt the CLS token on the target domain for efficient few-shot learning. Extensive experiments on four benchmarks validate our rationale and state-of-the-art performance. Our codes are available at https://github.com/Zoilsen/CLS_Token_CDFSL.

Cloud storage is widely used for flexible and efficient data management, yet redundant data across users requires storage optimization. Deduplication helps by storing only unique data, making data sharing and ownership verification essential post-deduplication. Current Proof of Ownership (PoW) methods rely on interactive communication, leading to delays and performance issues during intensive data operations, and often assume a level of trust that may not hold in practical scenarios. To overcome these issues, we propose ES-PoW, a non-interactive secure proof of ownership scheme for cloud storage. ES-PoW performs ownership verification in a single round, avoiding delays in block verification. Using modular exponentiation and the discrete logarithm problem, ES-PoW generates and verifies ownership proofs efficiently. Unlike previous schemes, ES-PoW is resilient against brute-force, replay, and Man-in-the-Middle attacks, without relying on trusted nodes, making it better suited to real-world applications. Experimental results show that ES-PoW reduces I/O operation time and accelerates computation, achieving up to 53.6× and 54.5× speed improvements over current methods.

Masked graph autoencoders have emerged as a powerful graph self-supervised learning method that has yet to be fully explored. In this paper, we unveil that the existing discrete edge masking and binary link reconstruction strategies are insufficient to learn topologically informative representations, from the perspective of message propagation on graph neural networks. These limitations include blocking message flows, vulnerability to over-smoothness, and suboptimal neighborhood discriminability. Inspired by these understandings, we explore non-discrete edge masks, which are sampled from a continuous and dispersive probability distribution instead of the discrete Bernoulli distribution. These masks restrict the amount of output messages for each edge, referred to as "bandwidths". We propose a novel, informative, and effective topological masked graph autoencoder using bandwidth masking and a layer-wise bandwidth prediction objective. We demonstrate its powerful graph topological learning ability both theoretically and empirically. Our proposed framework outperforms representative baselines in both self-supervised link prediction (improving the discrete edge reconstructors by at most 20%) and node classification on numerous datasets, solely with a structure-learning pretext. Our implementation is available at https://github.com/Newiz430/Bandana.

Recently, Oblivious Storage has been proposed to prevent privacy leakage from user access patterns, which obfuscates and makes it computationally indistinguishable from the random sequences by fake accesses and probabilistic encryption. The same data exhibits distinct ciphertexts. Thus, it seriously impedes cloud providers’ efforts to improve storage utilization to remove user redundancy, which has been widely used in the existing cloud storage scenario. Inspired by the successful adoption of removing duplicate data in cloud storage, we attempt to integrate obliviousness, remove redundancy, and propose a practical oblivious storage, PEO-Store. Instead of fake accesses, introducing delegates breaks the mapping link between a valid access pattern and a specific client. The cloud interacts only with randomly authorized delegates. This design leverages non-interactive zero-knowledge-based redundancy detection, discrete logarithm problem-based key sharing, and secure time-based delivery proof. These components collectively protect access pattern privacy, accurately eliminate redundancy, and prove the data delivery among delegates and the cloud. Theoretical proof demonstrates that, in our design, the probability of identifying the valid access pattern with a specific client is negligible. Experimental results show that PEO-Store outperforms state-of-the-art methods, achieving an average throughput of up to 3 times faster and saving 74% of storage space.

Bundle recommendation aims to recommend a bundle of items for the user to purchase together, for which two scenarios (i.e.Next-bundle recommendation and Within-bundle recommendation) are explored to recommend a specific bundle of items for the user and a specific item to fill the user's current bundle, respectively. Previous works largely model the user's preference with a uniform intent, without considering the diversity of intents when adopting the items within the bundle. In the real scenario of bundle recommendation, user intents modeling actually needs to be considered from three hierarchical levels, for that: a user's intents may be naturally distributed in different bundles (user level), one bundle may contain multiple intents of a user (bundle level), and an item in different bundles may also present different user intents (item level). To this end, we develop a novel model,HierarchicalIntentDisentangleGraphNetworks (HIDGN) for bundle recommendation. HIDGN is capable of capturing the diversity of the user's intent precisely and comprehensively from the hierarchical structure with an cross-task intent contrastive learning, which is unified with the supervised next-/within-bundle recommendation sub-tasks as a multi-task framework. Extensive experiments on three benchmark datasets demonstrate that HIDGN outperforms the state-of-the-art methods by 43.0%, 13.2%, and 73.3%, respectively.

The escalating proliferation of user generated contents such as videos and images are dominating the network traffic. The optimal strategy for mitigating backbone congestion and minimizing user request latency lies in prudent caching at edge stations within distributed autonomous networks, obviating the necessity to transmit data to the cloud. However, accurately caching content based on distributed autonomous networks requires elaborative collaboration between edge servers, which remains a great challenge, especially when content is highly dynamic and the storage resources of edge stations are limited. To tackle this challenge, this paper proposes a contextual bandit-based online caching algorithm for evaluating the optimal content hit rate reward, which can adapt to the constantly changing stream of emerging content. We build the content space, BS space, and a fine-grained space searching method to cache contents and corresponding edge stations. Furthermore, to perform collaborative caching and sharing between edges, we propose a federated autonomous multi-layer caching framework, whereby each server can locally learn the model for accurate caching and a synchronous mechanism is set up for global updating, further improving the hit rates. Finally, we perform theoretical proofs and simulations, demonstrating that our regret is sublinear and our caching algorithm outperforms several state-of-the-art algorithms.

This article studies the multimedia problem of temporal sentence grounding (TSG), which aims to accurately determine the specific video segment in an untrimmed video according to a given sentence query. Traditional TSG methods mainly follow the top-down or bottom-up framework and are not end-to-end. They severely rely on time-consuming post-processing to refine the grounding results. Recently, some transformer-based approaches are proposed to efficiently and effectively model the fine-grained semantic alignment between video and query. Although these methods achieve significant performance to some extent, they equally take frames of the video and words of the query as transformer input for correlating, failing to capture their different levels of granularity with distinct semantics. To address this issue, in this article, we propose a novelHierarchicalLocal-GlobalTransformer (HLGT) to leverage this hierarchy information and model the interactions between different levels of granularity and different modalities for learning more fine-grained multi-modal representations. Specifically, we first split the video and query into individual clips and phrases to learn their local context (adjacent dependency) and global correlation (long-range dependency) via a temporal transformer. Then, a global-local transformer is introduced to learn the interactions between the local-level and global-level semantics for better multi-modal reasoning. Besides, we develop a new cross-modal cycle-consistency loss to enforce interaction between two modalities and encourage the semantic alignment between them. Finally, we design a brand-new cross-modal parallel transformer decoder to integrate the encoded visual and textual features for final grounding. Extensive experiments on three challenging datasets (ActivityNet Captions, Charades-STA and TACoS) show that our proposed HLGT achieves a new state-of-the-art performance, demonstrating its effectiveness and computational efficiency.

Federated Learning (FL) is to allow multiple clients to collaboratively train a model while keeping their data locally. However, existing FL approaches typically assume that the data in each client is static and fixed, which cannot account for incremental data with domain shift, leading to catastrophic forgetting on previous domains, particularly when clients are common edge devices that may lack enough storage to retain full samples of each domain. To tackle this challenge, we proposeFederatedDomain-IncrementalLearning viaSynergisticReplay (SR-FDIL), which alleviates catastrophic forgetting by coordinating all clients to cache samples and replay them. More specifically, when new data arrives, each client selects the cached samples based not only on their importance in the local dataset but also on their correlation with the global dataset. Moreover, to achieve a balance between learning new data and memorizing old data, we propose a novel client selection mechanism by jointly considering the importance of both old and new data. We conducted extensive experiments on several datasets of which the results demonstrate that SR-FDIL outperforms state-of-the-art methods by up to 4.05% in terms of average accuracy of all domains.

Embedding tables are usually huge in click-through rate (CTR) prediction models. To train and deploy the CTR models efficiently and economically, it is necessary to compress their embedding tables. To this end, we formulate a novel quantization training paradigm to compress the embeddings from the training stage, termed low-precision training (LPT). Also, we provide theoretical analysis on its convergence. The results show that stochastic weight quantization has a faster convergence rate and a smaller convergence error than deterministic weight quantization in LPT. Further, to reduce accuracy degradation, we propose adaptive low-precision training (ALPT) which learns the step size (i.e., the quantization resolution). Experiments on two real-world datasets confirm our analysis and show that ALPT can significantly improve the prediction accuracy, especially at extremely low bit width. For the first time in CTR models, we successfully train 8-bit embeddings without sacrificing prediction accuracy.

Rationalization is to employ a generator and a predictor to construct a self-explaining NLP model in which the generator selects a subset of human-intelligible pieces of the input text to the following predictor.However, rationalization suffers from two key challenges, i.e., spurious correlation and degeneration, where the predictor overfits the spurious or meaningless pieces solely selected by the not-yet well-trained generator and in turn deteriorates the generator.Although many studies have been proposed to address the two challenges, they are usually designed separately and do not take both of them into account.In this paper, we propose a simple yet effective method named MGR to simultaneously solve the two problems.The key idea of MGR is to employ multiple generators such that the occurrence stability of real pieces is improved and more meaningful pieces are delivered to the predictor.Empirically 1 , we show that MGR improves the F1 score by up to 20.9% as compared to state-of-the-art methods.

Federated Distillation (FD) has recently attracted increasing attention for its efficiency in aggregating multiple diverse local models trained from statistically heterogeneous data of distributed clients. Existing FD methods generally treat these models equally by merely computing the average of their output soft predictions for some given input distillation sample, which does not take the diversity across all local models into account, thus leading to degraded performance of the aggregated model, especially when some local models learn little knowledge about the sample. In this paper, we propose a new perspective that treats the local data in each client as a specific domain and design a novel domain knowledge aware federated distillation method, dubbed DaFKD, that can discern the importance of each model to the distillation sample, and thus is able to optimize the ensemble of soft predictions from diverse models. Specifically, we employ a domain discriminator for each client, which is trained to identify the correlation factor between the sample and the corresponding domain. Then, to facilitate the training of the domain discriminator while saving communication costs, we propose sharing its partial parameters with the classification model. Extensive experiments on various datasets and settings show that the proposed method can improve the model accuracy by up to 6.02% compared to state-of-the-art baselines.

Federated Learning (FL) allows massive clients to collaboratively train a global model without revealing their private data. Because of the participants’ not independently and identically distributed (non-IID) statistical characteristics, it will cause divergence among the client’s Deep Neural Network model weights and require more communication rounds before training can be converged. Moreover, models trained from non-IID data may also extract biased features and the rationale behind the model is still not fully analyzed and exploited. In this paper, we propose eXplainable-Fed (XFed) which is a novel client selection mechanism that takes both accuracy and explainability into account. Specifically, XFed selects participants in each round based on a small test set’s accuracy via cross-entropy loss and interpretability via XAI-accuracy. XAI-accuracy is calculated by Intersection over Union Ratio between the heat map and the truth mask to evaluate the overall rationale of accuracy. The results of our experiments show that our method has comparable accuracy to state-of-the-art methods specially designed for accuracy while increasing explainability by 14%-35% in terms of rationality.

Knowledge distillation enhances the performance of the student model by transferring knowledge from the teacher model. Moreover, the attention mechanism has been introduced recently to enable each layer of the student to learn knowledge from all teacher layers, which brings about considerable optimization. However, noted that features from different layers, such as shallow and deep layers, might have a big semantic gap, and compulsively aligning one student layer to all teacher layers would mislead the learning process. To tackle this problem, an effective framework called Semantic Stage-Wise learning for Knowledge Distillation (SSWKD) is presented in this paper. We divide all layers into shallow and deep stages, and only allow feature alignment within the same stage to alleviate semantic mismatch. In addition, with the observation that the performance of deep networks relies more on some key features rather than evenly on all of them, a crucial feature enhancement method based on KL divergence is then proposed for SSWKD, forcing the student to pay more attention to critical features of the teacher. Extensive experiments and visualizations show that our SSWKD outperforms other distillation methods on CIFAR-100 and COCO2017 datasets for image classification, object detection, and instance segmentation tasks.

Graph Contrastive Learning (GCL) is an effective way to learn generalized graph representations in a self-supervised manner, and has grown rapidly in recent years. However, the underlying community semantics has not been well explored by most previous GCL methods. Research that attempts to leverage communities in GCL regards them as having the same influence on the graph, leading to extra representation errors. To tackle this issue, we define ''community strength'' to measure the difference of influence among communities. Under this premise, we propose a Community-Strength-enhanced Graph Contrastive Learning (CSGCL) framework to preserve community strength throughout the learning process. Firstly, we present two novel graph augmentation methods, Communal Attribute Voting (CAV) and Communal Edge Dropping (CED), where the perturbations of node attributes and edges are guided by community strength. Secondly, we propose a dynamic ''Team-up'' contrastive learning scheme, where community strength is used to progressively fine-tune the contrastive objective. We report extensive experiment results on three downstream tasks: node classification, node clustering, and link prediction. CSGCL achieves state-of-the-art performance compared with other GCL methods, validating that community strength brings effectiveness and generality to graph representations. Our code is available at https://github.com/HanChen-HUST/CSGCL.

Federated Learning enables collaboratively model training among a number of distributed devices with the coordination of a centralized server, where each device alternatively performs local gradient computation and communication to the server. FL suffers from significant performance degradation due to the excessive communication delay between the server and devices, especially when the network bandwidth of these devices is limited, which is common in edge environments. Existing methods overlap the gradient computation and communication to hide the communication latency to accelerate the FL training. However, the overlapping can also lead to an inevitable gap between the local model in each device and the global model in the server that seriously restricts the convergence rate of learning process. To address this problem, we propose a new overlapping method for FL, AOCC-FL, which aligns the local model with the global model via calibrated compensation such that the communication delay can be hidden without deteriorating the convergence performance. Theoretically, we prove that AOCC-FL admits the same convergence rate as the non-overlapping method. On both simulated and testbed experiments, we show that AOCC-FL achieves a comparable convergence rate relative to the non-overlapping method while outperforming the state-of-the-art overlapping methods.

A self-explaining rationalization model is generally constructed by a cooperative game where a generator selects the most human-intelligible pieces from the input text as rationales, followed by a predictor that makes predictions based on the selected rationales. However, such a cooperative game may incur the degeneration problem where the predictor overfits to the uninformative pieces generated by a not yet well-trained generator and in turn, leads the generator to converge to a sub-optimal model that tends to select senseless pieces. In this paper, we theoretically bridge degeneration with the predictor's Lipschitz continuity. Then, we empirically propose a simple but effective method named DR, which can naturally and flexibly restrain the Lipschitz constant of the predictor, to address the problem of degeneration. The main idea of DR is to decouple the generator and predictor to allocate them with asymmetric learning rates. A series of experiments conducted on two widely used benchmarks have verified the effectiveness of the proposed method. Codes: https://github.com/jugechengzi/Rationalization-DR.

Rationalization aims to strengthen the interpretability of NLP models by extracting a subset of human-intelligible pieces of their inputting texts. Conventional works generally employ the maximum mutual information (MMI) criterion to find the rationale that is most indicative of the target label. However, this criterion can be influenced by spurious features that correlate with the causal rationale or the target label. Instead of attempting to rectify the issues of the MMI criterion, we propose a novel criterion to uncover the causal rationale, termed the Minimum Conditional Dependence (MCD) criterion, which is grounded on our finding that the non-causal features and the target label are \emph{d-separated} by the causal rationale. By minimizing the dependence between the non-selected parts of the input and the target label conditioned on the selected rationale candidate, all the causes of the label are compelled to be selected. In this study, we employ a simple and practical measure for dependence, specifically the KL-divergence, to validate our proposed MCD criterion. Empirically, we demonstrate that MCD improves the F1 score by up to 13.7% compared to previous state-of-the-art MMI-based methods. Our code is in an anonymous repository: https://anonymous.4open.science/r/MCD-CE88.

The cost of hardware and energy consumption on training a click-through rate (CTR) model is highly prohibitive. A recent promising direction for reducing such costs is data distillation with gradient matching, which aims to synthesize a small distilled dataset to guide the model to a similar parameter space as those trained on real data. However, there are two main challenges to implementing such a method in the recommendation field: (1) The categorical recommended data are high dimensional and sparse one- or multi-hot data which will block the gradient flow, causing backpropagation-based data distillation invalid. (2) The data distillation process with gradient matching is computationally expensive due to the bi-level optimization. To this end, we investigate efficient data distillation tailored for recommendation data with plenty of side information where we formulate the discrete data to the dense and continuous data format. Then, we further introduce a one-step gradient matching scheme, which performs gradient matching for only a single step to overcome the inefficient training process. The overall proposed method is called Categorical data distillation with Gradient Matching (CGM), which is capable of distilling a large dataset into a small of informative synthetic data for training CTR models from scratch. Experimental results show that our proposed method not only outperforms the state-of-the-art coreset selection and data distillation methods but also has remarkable cross-architecture performance. Moreover, we explore the application of CGM on model retraining and mitigate the effect of different random seeds on the training results.

A common practice to keep the freshness of an offline Recommender System (RS) is to train models that fit the user’s most recent behaviour while directly replacing the outdated historical model. However, many feature engineering and computing resources are used to train these historical models, but they are underutilized in the downstream RS model training. In this paper, to turn these historical models into treasures, we introduce a model inversed data synthesis framework, which can recover training data information from the historical model and use it for knowledge transfer. This framework synthesizes a new form of data from the historical model. Specifically, we ’invert’ an off-the-shield pretrained model to synthesize binary class user-item pairs beginning from random noise without requiring any additional information from the training dataset. To synthesize informative data from a pretrained model, we propose a new continuous data type rather than the original one- or multi-hot vectors. An additional statistical regularization is added to further improve the quality of the synthetic data inverted from the deep model with batch normalization. The experimental results show that our framework can generalize across different types of models. We can efficiently train different types of classical Click-Through-Rate (CTR) prediction models from scratch with significantly few inversed synthetic data (2 orders of magnitude). Moreover, our framework can also work well in the knowledge transfer scenarios such as model retraining and data-free knowledge distillation.

With the increasing number of big data applications, large amounts of valuable data are distributed in different organizations or regions. Federated Learning (FL) enables collaborative model training without sharing sensitive data and is widely used in AI medical diagnosis, economy, and autonomous driving scenarios. However, it still leaks the privacy from the gradient exchange in federated learning. What’s worse, state-of-the-art work, such as Batchcrypt, still suffers from computational overhead due to a considerable amount of computation and communication costs caused by homomorphic encryption. Therefore, we propose a novel symmetric key-based homomorphic encryption scheme, Sym-Fed. To unleash the power of symmetric encryption in federated learning, we combine random masking with symmetric encryption and keep the homomorphic property during the gradient exchange in the federated learning process. Finally, the security analysis and experimental results on real workloads show that our design achieves performance improvement 6× to 668× and reduces the communication overhead 1.2× to 107× compared with the state-of-the-art work, BatchCrypt and FATE, without model accuracy degradation and security compromise.

Abstract The expansion of available information in location-based social networks (LBSNs) has led to information overload, making it urgent to discover users’ next point-of-interest (POI). Some existing works only consider certain modal information in LBSNs and do not transform them into high-dimensional structures, which hinders the alleviation of the data sparsity problem. Moreover, many approaches rely solely on social relationships, making it difficult to recommend POIs to new users without association information. To tackle these challenges, we propose a social- and spatial–temporal-aware next point-of-Interest (SSTP) recommendation model. SSTP uses two feature encoders based on self-attention mechanism and gate recurrent unit to model users’ check-in enhancement sequence hierarchically. We also design a random neighborhood sampling approach to mine user social relationships, thus alleviating the user cold start problem. Finally, we propose a geographical-aware graph attention network to learn the sensitivity of users to distance. Extensive experiments on two real-world datasets show that SSTP outperforms state-of-the-art models, improving Hit@k by 2.26–6.55 $$\%$$ % and MAP@k by 3.49–6.55 $$\%$$ % . Moreover, SSTP has better performance on sparse data, with an average improvement of 6.09 $$\%$$ % on the Hit@k. The code can be downloaded at https://github.com/Rih0/sstp .

With intelligent devices’ prevalence, mobile edge computing (MEC) technology can effectively process the massive data in the Internet of Things (IoT). The metadata produced in MEC plays a role in colossal data management. However, it also includes lots of user privacy information. Traditional privacy protection solutions sacrifice the metadata’s availability or are incompatible with the distributed environment. To overcome this dilemma, we propose a blockchain-based secure searching strategy for metadata (BSSMeta) in MEC. By leveraging a lightweight proxy re-encryption scheme and a decentralized key generation method on the blockchain, BSSMeta keeps the security of the metadata searching tasks. Meanwhile, a main/secondary smart contract method and a buffer uploading strategy are proposed for the efficiency of the searching and uploading processes. As a byproduct, BSSMeta also supports searching metadata in a multiuser environment and gives users access control to metadata. Finally, we provide a security analysis of BSSMeta and implement a prototype on the Hyperledger Fabric platform. The results of experiments on various workloads show that BSSMeta is feasible and flexible in security and efficiency.

Mobile-edge computing (MEC) is a new computing paradigm that enables cloud computing and information technology (IT) services to be delivered at the network’s edge. By shifting the load of cloud computing to individual local servers, MEC helps meet the requirements of ultralow latency, localized data processing, and extends the potential of the Internet of Things (IoT) for end-users. However, the crosscutting nature of MEC and the multidisciplinary components necessary for its deployment have presented additional security and privacy concerns. Fortunately, artificial intelligence (AI) algorithms can cope with excessively unpredictable and complex data, which offers a distinct advantage in dealing with sophisticated and developing adversaries in the security industry. Hence, in this article, we comprehensively provide a survey of security and privacy in MEC from the perspective of AI. On the one hand, we use European Telecommunications Standards Institute (ETSI) MEC reference architecture as our-based framework while merging the software-defined network (SDN) and network function virtualization (NFV) to better illustrate a serviceable platform of MEC. On the other hand, we focus on new security and privacy issues, as well as potential solutions from the viewpoints of AI. Finally, we comprehensively discuss the opportunities and challenges associated with applying AI to MEC security and privacy as possible future research directions.

Temporal sentence grounding (TSG) is an important yet challenging task in video-based information retrieval. Given an untrimmed video input, it requires the machine to predict the interested video segment semantically related to a given sentence query. Most existing TSG methods train well-designed deep networks to align the semantic between video-query pairs for activity grounding with a large amount of data. However, we argue that these works easily capture the selection biases of video-query pairs in a dataset rather than showing the robust reasoning abilities to handle the rarely appeared pairs (i.e., few-shot contents). To alleviate such limitation of the off-balance data distribution during the network training, in this paper, we propose a novel memory-augmented network called Memory-Guided Semantic Learning Network (MGSL-Net) to handle the few-shot TSG task for enhancing the model generalization ability. Specifically, given the matched video-query input, we first employ a graph attentive cross-modal interaction module to align their semantics in a cycle-consistent manner. Then, we develop the memory modules in both video and query domains to record the cross-modal shared semantic features in the domain-specific persistent memory. At last, a heterogeneous attention module is utilized to integrate the memory-enhanced multi-modal features in both video and query domains with further feature calibration. During training, the memory modules are dynamically associated with both common and rare cases to memorize all appeared contents, alleviating the issue of forgetting the few-shot contents. Therefore, in testing, the rare cases can be enhanced by retrieving the stored memories, improving the generalization ability of the model. Experimental results on three benchmarks (ActivityNet Caption, Charades-STA and TACoS) show the superiority of our method on both effectiveness and efficiency.

With the prevalence of cloud storage, people prefer to outsource their data to the cloud for flexibility and reliability. Undoubtedly, there are lots of redundancy among these data. However, high-end storage with deduplication costs heavy computation and increases the data management complexity. Potential customers need the redundancy proportion information of their outsourced data to decide whether high-end storage with deduplication is worthwhile. Thus, many researchers have previously attempted to predict the redundant ratio. However, existing mechanisms ignore the redundancy proportion among similar chunks containing many duplicate data. Although resemblance detection, detecting the duplicate parts among similar data, has become a hot issue, it is hardly applied to the conventional deduplication ratio estimation because of unacceptable calculation cost. Therefore, we analyze the limitations and challenges of deduplication ratio prediction in prediction scope and response time and further propose a novel prediction scheme. By leveraging the context-aware resemblance detection, and confidence interval theory, our method can achieve faster estimation speed with higher accuracy in deduplication ratio compared with the state-of-the-art work. Finally, the results show that our method can efficiently and effectively estimate the proportion of duplicate chunks and redundant data among similar chunks by conducting experiments on real workloads.

In this paper, we propose a novel chunk-context-aware resemblance detection al-gorithm called CARD. By introducing machine learning into deduplication, the chunk feature will embed the chunk-context information after the N-sub-chunk shingles based initial feature extraction and BP-Neural network training. In the predicting process, each chunk's initial feature corresponds to a chunk-context feature. Finally, the cloud calculates the different part among resemblance chunks based on these feature by delta encoding. Only the different part is stored. The basic workflow corresponds to Figure 1. For more detailed illustrations, please see our full paper here

Recently, cloud storage has been widely used in our daily life. And there are lots of redundancy among these outsourced data. Conventional deduplication technology efficiently splits these data at the chunk level and removes the duplicate chunks to save the network bandwidth and improve the cloud storage utility. But it ignores the redundancy among similar chunks. Resemblance detection has recently become a hot issue with detecting these redundant parts among similar data. CARD, the state-of-the-art work, can efficiently and effectively remove these redundancies by introducing the neural network with resemblance detection. However, the source domain of the CARD model may have an explicitly different input distribution. The cloud cannot deal with the possible future domain data based on CARD design. This cross-domain setting may serials degrades the performance of CARD. To overcome this problem, we propose a cross-domain resemblance detection scheme called MetaContext. Integrating the chunk-context aware model and the learn-to-learn idea can produce a more robust chunk feature than CARD. As a byproduct, it also outperforms the CARD in speed. Finally, we implement the MetaContext and conduct serial experiments on real workloads. The results show that our method can efficiently and effectively detect and remove the redundancy among similar data.

Rationalization aims to strengthen the interpretability of NLP models by extracting a subset of human-intelligible pieces of their inputting texts. Conventional works generally employ a two-phase model in which a generator selects the most important pieces, followed by a predictor that makes predictions based on the selected pieces. However, such a two-phase model may incur the degeneration problem where the predictor overfits to the noise generated by a not yet well-trained generator and in turn, leads the generator to converge to a suboptimal model that tends to select senseless pieces. To tackle this challenge, we propose Folded Rationalization (FR) that folds the two phases of the rationale model into one from the perspective of text semantic extraction. The key idea of FR is to employ a unified encoder between the generator and predictor, based on which FR can facilitate a better predictor by access to valuable information blocked by the generator in the traditional two-phase model and thus bring a better generator. Empirically, we show that FR improves the F1 score by up to 10.3% as compared to state-of-the-art methods.

Few-shot class-incremental learning (FSCIL) is designed to incrementally recognize novel classes with only few training samples after the (pre-)training on base classes with sufficient samples, which focuses on both base-class performance and novel-class generalization. A well known modification to the base-class training is to apply a margin to the base-class classification. However, a dilemma exists that we can hardly achieve both good base-class performance and novel-class generalization simultaneously by applying the margin during the base-class training, which is still under explored. In this paper, we study the cause of such dilemma for FSCIL. We first interpret this dilemma as a class-level overfitting (CO) problem from the aspect of pattern learning, and then find its cause lies in the easily-satisfied constraint of learning margin-based patterns. Based on the analysis, we propose a novel margin-based FSCIL method to mitigate the CO problem by providing the pattern learning process with extra constraint from the margin-based patterns themselves. Extensive experiments on CIFAR100, Caltech-USCD Birds-200-2011 (CUB200), and miniImageNet demonstrate that the proposed method effectively mitigates the CO problem and achieves state-of-the-art performance.

Summary Community detection is a fundamental research problem in social networks. However, most existing research focuses on homogeneous networks while ignoring the multitopology and attributes in social media. In this article, we propose community detection algorithms based on community kernels to detect high‐quality communities in heterogeneous social networks. It is noticed that the social community has multiple topology structures, as nodes or users in social media networks have multiple attributions. For example, users can be friends and coworkers in a research group simultaneously. Hence, we propose a multilayer and attribute combined measure (MACM), a novel measurement based on the multilayer structure and common neighboring attributes, which includes the similarity measure between nodes and the importance measure for individual node in multilayer networks. Two improved community kernel detection algorithms based on MACM are subsequently proposed. They are the MA‐Greedy, which is based on the greedy algorithm, and the MA‐WeBA, which is a weighted balanced algorithm. The multilayer structure and attributes are comprehensively considered when calculating the similarity and importance of nodes in these strategies. Extensive experimental results on two public data sets demonstrate that the multilayer structure and attribute information can be used to enhance the precision of community detection.

Abstract Chunking is one of the most important processes in a secure deduplication system. It determines the deduplication ratio, metadata size, and the encryption key number. Existing chunking algorithms can be categorized as client‐side chunking algorithm and the server‐aid chunking algorithm. Although the former one can directly be applied into the secure deduplication, it ignores the management overhead caused by the metadata size and encryption key number. Thus, the conventional server‐aid chunking algorithm is proposed. However, it is not secure because the interaction between the client and the server is based on the plaintext. Thus, we propose a two‐party secure server‐aid chunking algorithm, TSS. It supports the secure server‐aid interaction and reduces the metadata size and the encryption key number in the secure deduplication with a comparable deduplication ratio. The theoretical proof and experimental results show that our method outperforms the state‐of‐the‐art chunking algorithm, elastic chunking algorithm.

Federated Learning has attracted widespread attention in recent years because it allows massive edge nodes to collaboratively train machine learning models without sharing their private data sets. However, these edge nodes are usually heterogeneous in computational capability and statistically different in data distribution, i.e., non-independent and identically distributed (IID), leading to significant performance degradation. Although status quo asynchronous training methods can solve the heterogeneity issue, they cannot prevent the non-IID problem from reducing the convergence rate. In this article, we propose a novel paradigm that schedules the gradient with partially averaged gradients and applies the global momentum (GSGM) for asynchronous training over non-IID data sets in an edge environment. Our key idea is to apply global momentum and partial average on the biased gradients calculated on edge nodes after scheduling, to make the training process stable. Empirical results demonstrate that GSGM can well adapt to different degrees of non-IID data and bring 20% performance gains in terms of training stability for popular optimization algorithms with enhanced accuracy over Fashion-Mnist and CIFAR-10 data sets.

The ever-growing artificial intelligence (AI) applications have greatly reshaped our world in many areas, e.g., smart home, computer vision, natural language processing, etc. Behind these applications are usually machine learning (ML) models with extremely large size, which require huge data sets for accurate training to mine the value contained in the big data. Large ML models, however, can consume tremendous computing resources to achieve decent performance and thus, it is difficult to train them in resource-constrained Internet of Things (IoT) environments, which would prevent further development and application of AI techniques in the future. To deal with such challenges, there are many efforts on accelerating the training process for large ML models in IoT. In this article, we provide a comprehensive review on the recent advances toward reducing the computing cost during the training stage while maintaining comparable model accuracy. Specifically, the optimization algorithms that aim to improve the convergence rate are emphasized over various distributed learning architectures that exploit ubiquitous computing resources. Then, the article elaborates the computation hardware acceleration and communication optimization for collaborative training among multiple learning entities. Finally, the remaining challenges, future opportunities, and possible directions are discussed.

Distributed gradient descent has been widely adopted in the machine learning field because considerable computing resources are available when facing the huge volume of data. Specifically, the gradient over the whole data is cooperatively computed by multiple workers. However, its performance can be severely affected by slow workers, namely stragglers. Recently, coding-based approaches have been introduced to mitigate the straggler problem, but they could hardly deal with the heterogeneity among workers. Besides, they always discard the results of stragglers causing huge resource waste. In this article, we first investigate how to tolerate stragglers by discarding their results and then seek to leverage the stragglers. For tolerating stragglers, we propose a heterogeneity-aware coding scheme that encodes gradients adaptive to the computing capability of workers. Theoretically, this scheme is optimal for stragglers tolerance. Relying on the scheme, we further propose an algorithm called DHeter-aware to exploit the gradients of stragglers which we called delayed gradients. Moreover, theoretical results characterized for DHeter-aware exhibits the same convergence rate as the gradient descent without delayed gradients. Experiments on various tasks and clusters demonstrate that our coding scheme outperforms all the state-of-the-art methods and the DHeter-aware further accelerates the coding scheme by achieving 25 percent time savings.

With the growing popularity of cloud storage, how to prevent information leakage from cloud access patterns attracts great attention. Oblivious RAM is proposed for this purpose. It is designed for the memory system, and most existing work focused on improving performance in the main memory. Recently, ORAM has been extended to the cloud environment, and it is called Oblivious Data Storage. TaoStore, the state-of-the-art oblivious data storage system, integrates the ORAM technology with synchronous I/O technology to reduce the mean response time. As we observed, there is a strong locality existing in user accesses. However, existing Oblivious Storage research did not consider this. In this article, we propose Loco-Store, an oblivious data storage. In Loco-Store, we design a novel stash controller scheme that can dynamically group relevant blocks during the oblivious I/O processes. We also propose a locality-based eviction algorithm to keep the security guarantee. The theoretical proof proves that our scheme keeps the security definition of ORAM. Finally, we implement a prototype and conduct extensive experiments on real-world datasets. The results show that Loco-Store can save the network bandwidth consumption up to 39.19 percent, and reduce the overall access time by 26.17 percent

Although mobile crowdsensing (MCS) has become a green paradigm of collecting, analyzing, and exploiting massive amounts of sensory data, existing incentive mechanisms are not effective to stimulate users’s active participation and service contribution in multi-service exchange in MCS due to its specific features: a large number of heterogeneous users have asymmetric service requirements, workers have the freedom to choose sensing tasks as well as participation levels, and multiple sensing tasks have heterogeneous values which may be untruthful declared by the corresponding requesters. To address this issue, this article develops a green Stackelberg-game incentive mechanism to achieve selective fairness, truthfulness, and bounded efficiency while reducing the burden on the platform. First, we model the multi-service exchange problem as a Stackelberg multi-service exchange game consisting of multi-leader and multi-follower, in which each requester as a leader first chooses the reward declaration strategy and thus the payment for each sensing task, each worker as a follower then chooses the sensing plan strategy to maximize her own utility. We next introduce the concept of virtual currency to maintain the selective fairness to balance service request and service provision between users, in which a user earns/consumes virtual currency for providing/receiving services, and thus no one can always get services without providing services. Then, we present two novel algorithms to compute the unique Nash equilibrium for the sensing plan determination game and the reward declaration determination game, respectively, which together forms a unique Stackelberg equilibrium for the proposed game. Afterwards, we theoretically prove that the proposed green Stackelberg-game incentive mechanism achieves the desirable properties of selective fairness, truthfulness, bounded efficiency. Finally, extensive evaluation results are provided to support the validity and effectiveness of our mechanism compared with both baseline and theoretical optimal approaches.

Communication has been considered as a major bottleneck in large-scale decentralized training systems since participating nodes iteratively exchange large amounts of intermediate data with their neighbors. Although compression techniques like sparsification can significantly reduce the communication overhead in each iteration, errors caused by compression will be accumulated, resulting in a severely degraded convergence rate. Recently, the error compensation method for sparsification has been proposed in centralized training to tolerate the accumulated compression errors. However, the analog technique and the corresponding theory about its convergence in decentralized training are still unknown. To fill in the gap, we design a method named ECSD-SGD that significantly accelerates decentralized training via error-compensated sparsification. The novelty lies in that we identify the component of the exchanging information in each iteration (i.e., the sparsified model update) and make targeted error compensation over the component. Our thorough theoretical analysis shows that ECSD-SGD supports arbitrary sparsification ratio and achieves the same convergence rate as the non-sparsified decentralized training methods. We also conduct extensive experiments on multiple deep learning models to validate our theoretical findings. Results show that ECSD-SGD outperforms all the start-of-the-art sparsified methods in terms of both the convergence speed and the final generalization accuracy.

Android hybrid applications use the Web Kit engine on Android platform to render Web contents and process JavaScript codes, and enable JavaScript codes to access device sensitive resources and native APIs. Therefore, hybrid applications will bring different security issues compared to native applications. The existing privacy leak detection methods for Android native applications cannot be directly applied to hybrid applications. In addition to the Dalvik Virtual Machine (DVM), hybrid applications may also spread privacy in the WebView component, which increases the scope of privacy dissemination and increases the difficulty of taint tracking. We delve into the different characteristics of privacy spreading on the DVM, Webkit engine and JavaScript engine of hybrid ap-plications. We propose HTDroid, an efficient dynamic taint tracking system for hybrid applications. The core idea of HTDroid is to spread the taint tags in the DVM to the Web Kit engine, and add the function of tracking the spread of taint data to WebKit engine and JavaScript engine through source code instrumentation. Our experiments show that HTDroid can run on both Android emulators and devices and effectively detect privacy leaks in hybrid applications and can deal with the two ways of privacy leaks in hybrid applications while existing TaintDroid cannot. Compared to the original Android system, HTDroid incurs only 22% performance overhead on CaffenieMark benchmark and impose 4.3 % overhead on J avaScript V8 engine benchmark, which is acceptable for run-time taint tracking.

With the progress of power energy storage technologies in capacity, cycle life and reliability, data center can optimize its utilization of energy storage battery to reduce its Total Cost of Ownership (TCO). Data center can cut the peak and fill the valley of their power consumption graphs with proper management of battery charging and discharging, while maintaining uninterruptible power system (UPS) capacity. This paper studies the control technology of data center battery charging and discharging based on Deep Reinforcement Learning (DRL). According to the electricity price and status and cycle life of batteries, the appropriate time is selected to charge and discharge batteries in order to maximize the electricity bill savings. To achieve higher benefits, the system state, charging and discharging actions, reward function and a neural network structure were designed in detail. According to the simulation results, the proposed algorithm can infer the best savings strategy in both USA and Beijing electricity price systems. Compared with the baseline algorithm, the priority experience playback Deep Q-network (DQN) can increase the energy storage savings up to 47% and 55% with the electricity prices of the USA and Beijing respectively.

Word sense disambiguation is a challenging task that aims to distinguish the correct sense for a target word. Unlike typical methods which only use the context information, we present the basic idea that incorporates the global sense distribution and contextual sense dependency to determine the correct sense. In this paper, we leverage the generative process of the sequence topic model to model the sense distribution from the whole global corpus directly. Since contextual sense dependency is more likely to happen between successive words, we hypothesize that the sense assignment of the target word depends on the sense of the previous word. Hence, the sense dependency could be modeled by the probabilistic topic model with hidden Markov chain assumption. Furthermore, the information in the sense inventory of WordNet is used as prior knowledge for obtaining the non-uniform sense distribution over words. It is notable that the prior knowledge contributes to parameter learning and inference during the Gibbs Sampling procedure, hence we call the proposed method a knowledge-based unsupervised approach. We evaluate the proposed method on Senseval-2, Senseval-3, SemEval-2007, SemEval-2013, and SemEval-2015 English All-Word WSD datasets and the experimental results show that although the performance of the proposed method is restricted by the sparse problem, it still achieves comparable performance compared with the state-of-the-art knowledge-based approaches.

Inference process of the semantic segmentation is a dense predictive task who requires heavy computational cost. However, the available computational resource is quite limited in many practical scenarios. Therefore, real-time semantic segmentation balancing both speed and accuracy is becoming particularly significant. In this paper, we propose a resolution-preserving framework for real-time semantic segmentation. A lightweight backbone network is employed to obtain high-level semantic features, which will be fully reused. A resolution-preserving module is designed to extract low-level spatial details while retaining high-resolution. We also introduce a feature refinement module to improve the performance. Experiments on Cityscapes dataset demonstrate that the proposed ERPNet achieves a balance between speed and accuracy. Especially, we obtain the results of 73.6% mIoU on the Cityscapes test set with the speed of 109.5 FPS on single GeForce GTX 2080 card.

With the prevalence of cloud storage, data deduplication has been a widely used technology by removing cross users’ duplicate data and saving network bandwidth. Nevertheless, traditional data deduplication hardly detects duplicate data among resemblance chunks. Currently, a resemblance data deduplication, called Finesse, has been proposed to detect and remove the duplicate data among similar chunks efficiently. However, we observe that the chunks following the similar chunk have a high chance of resembling data locality property, and vice versa. Processing these adjacent similar chunks in small average chunk size level increases the metadata, which deteriorates the deduplication system performance. Moreover, existing resemblance data deduplication schemes ignore the performance impact from metadata. Therefore, we propose a fast variable-grained resemblance data deduplication for cloud storage. It dynamically combines the adjacent resemblance chunks or unique chunks or breaks those chunks, located at the transition region between resemblance chunks and unique chunks. Finally, we implement a prototype and conduct a serial of experiments on real-world datasets. The results show that our method dramatically reduces the metadata size while achieving the high deduplication ratio.

Summary The amount of outsourced data grows rapidly. In recent years, cloud service providers integrate data deduplication systems with convergent encryption (CE) methods, in which a file encryption key is determined by its own content instead of the secret of a specific user, to save the storage cost and ensure the security of outsourced data. However, present secure deduplication systems failed to deal with data modifications efficiently. We observe that when a client makes small changes on an existing file, the current chunking algorithms cannot effectively detect the similarities and always create chunks with largely overlapped contents. It reduces data deduplication ratios and results in unnecessary overhead. In this paper, we propose Sed‐Dedup, an efficient secure delta encoding deduplication system to address this problem. In Sed‐Dedup, we introduce a novel delta encoding approach to store modified contents in delta files and leave the original files intact. Two schemes with different encoding policies are designed. Both of them can solve the issue and improve the secure deduplication performance. To evaluate the performance, we implement a prototype and conduct extensive experiments based on synthetic and real‐world datasets. Our experimental results show that Sed‐Dedup is superior to the state‐of‐the‐art secure deduplication systems.

When running in Parameter Server (PS), the Distributed Stochastic Gradient Descent (D-SGD) incurs significant communication delays and huge communication overhead due to the model synchronization. Moreover, considering the heterogeneity of computational capability among workers, traditional synchronization modes incur under-utilization of computational resources because fast workers have to wait for slow ones finishing the computation. Although our previous work OSP can effectively solve these problems by overlapping the computation and communication procedures and allowing adaptive multiple local updates in distributed training, it causes the staleness problem brought by the overlap, yielding a performance degradation. In this paper, we propose a new method named LOSP by introducing local compensation to our previous synchronization mechanism, which mitigates adverse effects caused by the overlapping synchronization. We theoretically prove that LOSP (1) preserves the same convergence rate as the sequential SGD for non-convex problems, and (2) exhibits good scalability due to the linear speedup property with respect to both the number of workers and the average number of local updates. Evaluations show that LOSP significantly improves performance over the state-of-the-art ones in terms of both convergence accuracy and communication cost.

Event-oriented news retrieval (ENR) is the task of retrieving news articles related to the specific event in response to the event-oriented query. Previous approaches usually focus on optimizing traditional retrieval models through hand-crafted features from the perspective of new articles. However, these approaches often fail to work well in reality, as they do not consider the essential natures of the event, i.e., dynamics, coupling. In this paper, we propose a novel and effective event-oriented neural ranking model for news retrieval (ENRMNR). Our model exploits a deep attention mechanism to tackle the dynamics and coupling derived from event evolution. Specifically, the word-level bidirectional attention allows the model to identify which query words about the subevent are related to the news article words, and vice-versa, in order to tackle the dynamics. Moreover, the hierarchical attention at passage-level and document-level allows it to capture fine-grained event representations for the coupling between different events within a news article. Experimental results on real-world datasets demonstrate that ENRMNR model significantly outperforms competitive models.

Existing gradient compression schemes fail to automatically determine the compression ratio or are accompanied by high compression overhead. To address this, we present Tail, an automated and lightweight gradient compression framework stacked by three modules, quantization, sparsification, and encoding. Without any hand-tuned effort, quantization module automatically adjusts the compression ratio along training iterations to retain accuracy first. Then, sparsification and encoding modules are successively applied to the quantized gradient to further improve compression ratio. Moreover, Tail reduces the compression overhead by approximate computing in the automated decision-making process. Experiments validate that Tail can reduce communication traffic by an order of magnitude while retaining or even improving model accuracy.

To accelerate distributed deep learning, gradient quantization technique is widely used to reduce the communication cost. However, the existing quantization schemes suffer from either model accuracy degradation or low compression ratio (arisen from a redundant setting of quantization level or high overhead in determining the level). In this work, we propose a novel adaptive quantization scheme (AdaQS) to explore the balance between model accuracy and quantization level. AdaQS determines the quantization level automatically according to gradient's mean to standard deviation ratio (MSDR). Then, to reduce the quantization overhead, we employ a computationally-friendly way of moment estimation to calculate the MSDR. Finally, theoretical analysis of AdaQS's convergence is conducted for non-convex objectives. Experiments demonstrate that AdaQS performs excellently on very deep model GoogleNet with 2.55% accuracy improvement relative to vanilla SGD and achieves 1.8x end-to-end speedup on AlexNet in a distributed cluster with 4*4 GPUs.

When expressing sentiment towards products, customers often explicitly indicate their recommendation status. Nevertheless, most existing literature focuses on sentiment analysis but neglects the rich correlation information that may be brought by explicit recommendation classification. We argue that the two tasks are correlated and hence, the knowledge in explicit recommendation classification can also be beneficial to sentiment analysis. Consequently, in this paper, a novel bidirectional encoder representations from transformers (BERT)-enhanced multi-task learning (BeMTL) approach is proposed to improve sentiment analysis with explicit recommendation classification. Specifically, the proposed MTL approach takes contextualized word embeddings produced by the pre-trained BERT-based embedding layer. Then, it learns the sentence contextual features shared between both tasks with a convolutional multi-head attention neural network. To fully exploit the correlation information between sentiment analysis and explicit recommendation classification tasks, a novel inter-task matching layer (IML) is designed to match their representations. In nutshell, our study reveals the potential of multi-task learning models on such types of problems, and experimental results on two Amazon datasets show that our approach outperforms the state-of-the-art baseline approaches for sentiment analysis.

App-level virtualization becomes increasingly popular. It allows multiple instances of an application to run simultaneously on the same Android system, without requiring modification of the Android firmware. These virtualization-capable apps are used by more than 100 million users worldwide. We conduct a systematic study of the implementation of app-level virtualization and the security threats that their users may face. First, we survey more than 160 apps collected from several popular app markets which can provide application virtualization capability. We find that these apps are implemented based on a similar design, and apps running in such a virtual environment are not completely isolated from each other. Second, we analyze malicious virtualized guest apps, and identify several areas of potential attack vectors, including privilege escalation, code injection, ransomware, etc. Malicious virtualized guest apps can launch reference hijacking attacks. Once a legitimate app is running in the virtual context, all of its sensitive data will be exposed to the host app. Third, we find a new type of repackaging attack. In our collection of 2 million app data set, we find that 68 apps pack and load malwares by using the virtualization technology to evade antivirus detection, 91 apps pack some legal apps for the purpose of wide distribution, and insert screen ads to gain profits at its startup. Finally, we discuss a variety of mitigation solutions for users, developers and vendors.

Industry 4.0 aims to create a modern industrial system by introducing technologies, such as cloud computing, intelligent robotics, and wireless sensor networks. In this article, we consider the multichannel access and task offloading problem in mobile-edge computing (MEC)-enabled industry 4.0 and describe this problem in multiagent environment. To solve this problem, we propose a novel multiagent deep reinforcement learning (MADRL) scheme. The solution enables edge devices (EDs) to cooperate with each other, which can significantly reduce the computation delay and improve the channel access success rate. Extensive simulation results with different system parameters reveal that the proposed scheme could reduce computation delay by 33.38% and increase the channel access success rate by 14.88% and channel utilization by 3.24% compared to the traditional single-agent reinforcement learning method.

In the real-world social networks, hashtags are widely applied for understanding the content of an individual microblog. However, users do not always take the initiative in attaching hashtags when posting a microblog so that much effort has been invested for automatically hashtag recommendation. As a new trend, users no longer only post texts but prefer to share with multimodal data, such as images. To deal with these situations, we propose an attention-based multimodal neural network model (AMNN) to learn the representations of multimodal microblogs and recommend relevant hashtags. In this article, we convert the hashtag recommendation task into a sequence generation problem. Then, we propose a hybrid neural network approach to extract the features of both texts and images and incorporate them into the sequence-to-sequence model for hashtag recommendation. Experimental results on the data set collected on Instagram and two public data sets demonstrate that the proposed method outperforms state-of-the-art methods. Our model achieves the best performance in three different metrics: precision, recall, and accuracy. The source code of this article can be obtained from “https://github.com/w5688414/AMNN.”

In recent years, the heterogeneous distributed platform integrating with FPGAs to accelerate computation tasks has been widely studied to deal with the deluge of data. However, most of current works suffer from poor universality and low resource utilization that run specific algorithms with the highly customized structure. Moreover, there are still many challenges, such as data curation, task scheduling, and resource management, which further limit the scalability of a CPU-FPGA distributed platform. In this paper, we present HeteroYARN, an FPGA-accelerated heterogeneous architecture based on YARN platform, which provides resource management and programming support for computing-intensive applications using FPGAs. In particular, the HeteroYARN abstracts FPGA accelerators as general resources and provides programming APIs to utilize those accelerators easily. Our HeteroYARN simplifies the request and usage of FPGA resources to enhance the efficiency of the heterogeneous framework while maintaining previous workflow unchanged. Experimental results using two representative algorithms, K-means and Naive Bayes classifier, which are accelerated by FPGAs, demonstrate the usability of the HeteroYARN framework and show performance speedup improvement by 7.5x (K-means) and 2.3x (Naive Bayes) respectively compared to conventional CPU-only applications provided by Mahout.

Logistic regression (LR) is the most commonly used loss function in multi-label image classification. However, it suffers from class imbalance problem caused by the huge difference in quantity between positive and negative samples as well as between different classes. First, we find that feeding randomly generated noise samples into an LR classifier is an effective way to detect class imbalances, and further define an informative imbalance metric named inference tendency based on noise sample analysis. Second, we design an efficient moving average based method for calculating inference tendency, which can be easily done during training with negligible overhead. Third, two novel rectification methods called extremum shift (ES) and tendency constraint (TC) are designed to offset or constrain inference tendency in the loss function, and mitigate class imbalances significantly. Finally, comparative experiments with Resnet on Microsoft COCO, NUS-WIDE and DeepFashion demonstrate the effectiveness of inference tendency and the superiority of our approach over the baseline LR and several state-of-the-art alternatives.

With the implementation of mainstream DL frameworks, scarce GPU memory resource is the primary bottleneck that hinders the trainability and training efficiency of ultra-deep neural networks (UDNN). Prior memory optimization works focus on removing the trainability restriction but leave the training efficiency out of consideration. To fill the gap, we present "AccUDNN", an accelerator that aims to make full use of finite GPU memory resource to speed up the training process of UDNN in this paper. AccUDNN mainly includes two modules: memory optimizer and hyperparameter tuner. Memory optimizer develops a novel performance-model guided dynamic swap out/in strategy to meet trainability first and further remedy the efficiency degradation in other swapping strategies. Then, a hyperparameter tuner is designed to explore the efficiency-optimal minibatch size and the matched learning rate after applying the dynamic swapping strategy. Evaluations demonstrate that AccUDNN cuts down the GPU memory requirement of ResNet-152 from more than 24GB to 8GB. In turn, given 12GB GPU memory budget, the efficiency-optimal minibatch size can reach 4.2x larger than Caffe and finally improve the scaling efficiency (speedup) of 8 GPUs' cluster by 1.9x.

Gradient descent algorithms are widely used in machine learning. In order to deal with huge volume of data, we consider the implementation of gradient descent algorithms in a distributed computing setting where multiple workers compute the gradient over some partial data and the master node aggregates their results to obtain the gradient over the whole data. However, its performance can be severely affected by straggler workers. Recently, some coding-based approaches are introduced to mitigate the straggler problem, but they are efficient only when the workers are homogeneous, i.e., having the same computation capabilities. In this paper, we consider that the workers are heterogenous which are common in modern distributed systems. We propose a novel heterogeneity-aware gradient coding scheme which can not only tolerate a predetermined number of stragglers but also fully utilize the computation capabilities of heterogenous workers. We show that this scheme is optimal when the computation capabilities of workers are estimated accurately. A variant of this scheme is further proposed to improve the performance when the estimations of the computation capabilities are not so accurate. We conduct our schemes for gradient descent based image classification on QingCloud clusters. Evaluation results show that our schemes can reduce the whole computation time by up to 3× compared with a state-of-the-art coding scheme.

The traditional convolutional neural networks (CNNs) learn numbers of fixed kernels (filters), which are used to obtain the representations of fixed patterns. Therefore, the knowledge representations of CNNs are limited to the number of kernels. In this paper, we present a Self-Representation Convolutional (SRC) layer to obtain richer knowledge representations of images by fully considering the self-similarity between adjacent pixels. SRC layer comprises a learnable local correlation measurement which measures the importance of adjacent pixels to the current pixel and two learnable linear parameters that perform linear projection on adjacent pixels and the weighted sum vectors, respectively. Compared with regular convolutional layers, the SRC layers can not only obtain comparable knowledge representations, but also reduce by a factor of 3× to 56× in the number of learnable parameters. Empirically, CNNs with SRC layers, called Self-Representation Convolutional Neural Networks (SRCNN), achieve strong performances on a range of visual datasets (SVHN, CIFAR-10 and CIFAR-100) while enjoying significant parameters and FLOPs savings.

When running in Parameter Server (PS), the Distributed Stochastic Gradient Descent (SGD) incurs significant communication delays because after pushing their updates, computing nodes (workers) have to wait for the global model to be communicated back from the master in every iteration. In this paper, we devise a new synchronization parallel mechanism named overlap synchronization parallel (OSP), in which the waiting time is removed by conducting computation and communication in an overlapped manner. We theoretically prove that our mechanism could achieve the same convergence rate compared to the sequential SGD for non-convex problems. Evaluations show that our mechanism significantly improves performance over the state-of-the-art ones, e.g., by 4× for both AlexNet and ResNet18 in terms of convergence speed.

Point-of-interest (POI) recommendation is one of the most important services in the rapid growing location-based social networks (LBSNs). Good POI recommendation can help people explore the locations they haven't visited but are interested in, and help merchants find their target users. Time-aware POI recommendation aims to recommend unvisited POIs for a given user at a specified time in a day. However, previous methods, such as user-based collaborative filtering, lack the mining of the features of POIs and the learning of abstract spatio-temporal interactions. In this paper, we propose a novel time-aware POI recommendation method named ST-RNet (Spatio-Temporal Recommender Network) to address these shortages. ST-RNet works in the following fashion. Firstly, we analyze the crucial features in LBSNs to alleviate data sparsity problem and further measure the similarities between POIs. For subsequent network training, we then construct the embedding matrices with same dimension for users and POIs by POI-based Collaborative Filtering (PCF). Furthermore, the positive and negative check-in records are fed into a novel recommender neural network (RNet) to learn the embedding matrix of times and the abstract interactions between users, POIs and times. Finally, ST-RNet recommends the unvisited POIs most likely to be visited to a given user at a given time. The experimental results on Foursquare real-world dataset show that ST-RNet is effective on time-aware POI recommendation task and is capable of analyzing the hidden patterns behind spatio-temporal interactions.

Recognizing text from natural images is a challenging and hot research topic in computer vision, yet not completely solved. The recent methods regard this task as a sequence labeling problem. In this task, there is a strong correspondence between the position of the input image patches sequence and the output character sequence. However, most of the recent recognition systems rarely consider this local information of the input sequence when recognizing the current character. In contrast to this, we present a Local Restricted Attention (LRA) mechanism to encode the current vector by considering adjacent vectors of the input sequence. We propose an ensemble decoder block which combines LRA mechanism with a regular decoder mechanism. This block not only brings significant improvement of recognition results under shorter training time but also can be easily embedded in other recognition frameworks. In addition, we propose a scene text recognition network based on the ensemble decoder. The experimental performances show that the proposed model achieves the state-of-the-art on several benchmark datasets including IIIT-5K, SVT, CUTE80, SVT-Perspective and ICDARs.

Session-based recommendation which aims to recommend the next item in an anonymous session for users, becomes one of the most popular tasks in recommendation area. Traditional methods such as matrix factorization and item-to-item perform very poorly because they only take into account the last click of the session and ignore the information of the whole click sequence. On the other hand, Recurrent Neural Network (RNN) based methods have performed excellently in session-based recommendation. However, they only consider the user's sequential behavior in the current session or only use cross-session information to track user's interests over time, whereas the user preference is not emphasized in cross-session. Therefore, in this work, we design a novel neural network framework for personalized session-based recommendation, named Dual Attentive Neural Network (DANN). DANN considers user's main purpose of current session and user's personalized preference of cross-session. Specifically, in DANN we exploit a user-level attention mechanism to model user's personalized preference and capture user's main purpose in the current session via a session-level attention mechanism. The experimental results on two real-world datasets show that our DANN model outperforms other baseline models. Furthermore, we find that DANN achieves improvement when modeling user personalized preferences, which shows the advantage of modeling user's preference and user's purpose simultaneously.

With the rapidly growing number of connected smart devices deployed and diverse services provided in the Internet of Things (IoT), service selection system, which discovers appropriate services for users, is becoming more and more important. However, challenges exist as a result of the highly heterogeneous environments, characteristics of various kinds of users and the expanding number of services offered by many service providers, which have promising applications in IoT. In the meantime, users' contexts (e.g., location, time, and surroundings), wildly utilized in the IoT scenario to better satisfy individuals' demands, raises privacy issues. To address these problems, we propose a privacy-preserving mobile edge computing (MEC) enabled context-aware service selection system over software defined networks (SDN) in IoT to provide suitable services for end users, where the edge nodes (ENs) cache and process information, acting as cooperative learners. Utilizing the users' feedback and the historical records monitored by the SDN-based network, our contextual online learning algorithm achieves high prediction accuracy. Besides, instead of considering them as individual items, we utilize a top-down cover tree structure to handle service data, which supports ever-increasing large-scale datasets and complex situations. We theoretically prove that the accumulative regret of our algorithm has a sublinear bound and our numerical results confirm that our algorithm can handle big data problems while achieving a balance between privacy-preserving level and service selection accuracy.

Ultra-deep neural network(UDNN) tends to yield higher-quality model but its training process is often difficult to handle. Scarce GPU DRAM capacity is the primary bottleneck that limits the depth of neural network and the range of trainable minibatch size. In this paper, we present a scheme that dedicates to make the utmost use of finite GPU memory resource to speed up the training process for UDNN. Firstly, a performance-model guided dynamic swap out/in strategy between GPU and host memory is carefully orchestrated to tackle the out-of-memory problem without introducing performance penalty. Then, a hyperparameter (minibatch size, learning rate) tuning policy is designed to explore the optimal configuration after applying the swap strategy from the perspectives of training time and final accuracy simultaneously. Finally, we verify the effectiveness of our scheme in both single and distributed GPU mode.

Predicting users' irregular trips in a short term period is one of the crucial tasks in the intelligent transportation system. With the prediction, the taxi requesting services, such as Didi Chuxing in China, can manage the transportation resources to offer better services. There are several different transportation scenes, such as commuting scene and entertainment scene. The origin and the destination of entertainment scene are more unsure than that of commuting scene, so both origin and destination should be predicted. Moreover, users' trips on Didi platform is only a part of their real life, so these transportation data are only few weak samples. To address these challenges, in this paper, we propose Taxi Demand Prediction (TDP) model in challenging entertainment scene based on heterogeneous graph embedding and deep neural predicting network. TDP aims to predict next possible trip edges that have not appeared in historical data for each user in entertainment scene. Experimental results on the real-world dataset show that TDP achieves significant improvements over the state-of-the-art methods.

Energy-saving (ES) systems developed on the basis of the Internet-of-Things (IoT) by heavily relying on automated understanding of human behaviors and activities recognition is of paramount importance in smart home. However, classic approaches are incapable to understand the relations among users' contexts and ES of appliances very well, and they cannot handle massive metering and time-varying user context datasets. Moreover, privacy concern is thoroughly aroused from both the residential and utility provider sides as to its essentiality. To tackle these problems, we propose a privacy-preserving and residential context-aware online ES (PRCOES) system in an IoT-enabled smart home environment. We model the repeated interaction of ES of appliances and the activity recognition of user context as a contextual multiarmed bandits (CMAB) problem, where the context-aware online learning algorithm can predict appropriate energy offers (EOs) that could meet the users' satisfaction, task completion rate, and ES purposes for appliances. We utilize a tree-based structure expanding from top to bottom to recommend EOs, which supports ever-increasing big metering datasets with user context-awareness. Theoretical analysis shows that our proposal achieves sublinear regret and differential privacy for both residents and utility provider. Experiments results validate that PRCOES could enhance users' experience and prolong users' engagement in everyday ES while guarantee the privacy for both residents and utility provider.

A logic program with function symbols is called finitely ground if there is a finite propositional logic program whose stable models are exactly the same as the stable models of this program. Finite groundability is an important property for logic programs with function symbols because it makes feasible to compute such programs' stable models using traditional ASP solvers. In this paper, we introduce new decidable classes of finitely ground programs called poly-bounded and k-EXP-bounded programs, which, to the best of our knowledge, strictly contain all other decidable classes of finitely ground programs discovered so far in the literature. We also study the relevant complexity properties for these classes of programs. We prove that the membership complexities for poly-bounded and k-EXP-bounded programs are EXPTIME-complete and (k+1)-EXPTIME-complete, respectively.

Android usually employs the Secure Socket Layer (SSL) protocol to protect the user’s privacy in network transmission. However, developers may misuse SSL-related APIs, which would lead attackers to steal user’s privacy through man-in-the-middle attacks. Existing methods based on static decompiling technology to detect SSL security vulnerabilities of Android applications cannot cope with the increasingly common packed applications. Meanwhile, dynamic analysis approaches have the disadvantages of excessive resource consumption and time-consuming. In this paper, we propose a dynamic method to solve this issue based on our novel automatic traversal model. At first, we propose several new traversal strategies to optimize the widget tree according to the user interface (UI) types and the interface state similarity. Furthermore, we develop a more granular traversal model by refining the traversal level from the Activity component to the Widget and implement a heuristic depth-first traversal algorithm in combination with our customized traversal strategy. In addition, the man-in-the-middle agent plug-in is extended to implement real-time attack test and return the attack results. Based on the above ideas, we have implemented SSLDetecter, an efficient automated detection system of Android application SSL security vulnerability. We apply it on multiple devices in parallel to detect 2456 popular applications in several mainstream application markets and find that 424 applications are suffering from SSL security vulnerabilities. Compared with the existing system SMV-HUNTER, the time efficiency of our system increases by 38% and the average detection rate increases by 6.39 percentage points, with many types of SSL vulnerabilities detected.

With the popularity of cloud storage, how to verify the integrity of data on the cloud has become a challenging problem. Traditional verification framework involves the Third Party Auditors (TPAs) which are not entirely credible. In this paper, we present a framework for blockchain-based data integrity verification in P2P cloud storage, making verification more open, transparent, and auditable. In this framework, we present Merkle trees for data integrity verification, and analyze the system performance under different Merkle trees structures. Furthermore, we develop rational sampling strategies to make sampling verification more effective. Moreover, we discuss the optimal sample size to tradeoff the conflict between verification overhead and verification precision, and suggest two efficient algorithms of order of verification. Finally, we conduct a series of experiments to evaluate the schemes of our framework. The experimental results show that our schemes can effectively improve the performance of data integrity verification.

Summary As social networks such as micro‐blogging sites rapidly grow, deciding whom to follow (followee recommendation) becomes a significantly important problem. Most existing works exclusively rely on two traditional factors: the proximity between two users in the network topology or the similarity of the user‐generated contents in the social network, disregarding the effect of users' following behaviors. The challenge of how to effectively combine these two factors remains largely open. Moreover, most research studies simply sort the scores to find top‐k users, which is time‐consuming, especially for large‐scale networks. In this paper, we propose the idea that “predict users' following behaviors by following behaviors themselves.” We consider a user's following to others as a normal process of dynamic and coherent behavior, and we model the potential propagation of the users' following behaviors. Furthermore, based on our previous research on top‐k selection problem, we propose an effective top‐k followee recommendation algorithm, called FRFB. FRFB has low complexity and high scalability and, moreover, good adaptability to real‐life dynamic social networks. We conduct extensive experiments, with two real social network data sets (Wiki and Twitter), which show that FRFB outperforms the well‐known topology‐based followee recommendation algorithms.

With the popularity of cloud computing, mobile devices can store/retrieve personal data from anywhere at any time. Consequently, the data security problem in mobile cloud becomes more and more severe and prevents further development of mobile cloud. There are substantial studies that have been conducted to improve the cloud security. However, most of them are not applicable for mobile cloud since mobile devices only have limited computing resources and power. Solutions with low computational overhead are in great need for mobile cloud applications. In this paper, we propose a lightweight data sharing scheme (LDSS) for mobile cloud computing. It adopts CP-ABE, an access control technology used in normal cloud environment, but changes the structure of access control tree to make it suitable for mobile cloud environments. LDSS moves a large portion of the computational intensive access control tree transformation in CP-ABE from mobile devices to external proxy servers. Furthermore, to reduce the user revocation cost, it introduces attribute description fields to implement lazy-revocation, which is a thorny issue in program based CP-ABE systems. The experimental results show that LDSS can effectively reduce the overhead on the mobile device side when users are sharing data in mobile cloud environments.

In the social network, to users, the tag is an important basis to mark and classify the resource. The tag of microblogging users can be used for advertising and network marketing. This paper presents a method based on naive Bayesian to predict the user tag. We use the user's basic attributes and some popular public tags as the features in Bayesian to predict whether a public tag belongs to a user. The experimental results show that the proposed method can achieve 87% accuracy.

Data chunking is one of the most important issues in a deduplication system, which not only determines the effectiveness of deduplication such as deduplication ratio, but also impacts the modification overhead. It breaks the file into chunks to find out the redundancy by fingerprint comparisons. The content-defined chunking algorithms such as TTTD, BSW CDC, and RC, can resist the boundary shift problem caused by small modifications. However, we observe that there exist a lot of consecutive maximum chunk sequences in various benchmarks. These consecutive maximum chunk sequences will lead to local boundary shift problem when facing small modifications. Based on this observation, we propose a new chunking algorithm, Elastic Chunking. By leveraging dynamic adjustment policy, elastic chunk can quickly find the boundary to remove the consecutive maximum chunk sequences. To evaluate the performance, we implement a prototype and conduct extensive experiments based on synthetic and realistic datasets. Compared with TTTD, BSW CDC and RC algorithms, proposed chunking algorithm can achieve the higher deduplication ratio and throughput.

Sorting is a basic computational task in Computer Science. As a variant of the sorting problem, top-k selecting have been widely used. To our knowledge, on average, the state-of-the-art top-k selecting algorithm Partial Quicksort takes C(n, k) = 2(n+1)Hn+2n-6k+6-2(n+3-k)Hn+1-k comparisons and about C(n, k)/6 exchanges to select the largest k terms from n terms, where Hn denotes the n-th harmonic number. In this paper, a novel top-k algorithm called DC-Top-k is proposed by employing a divide-and-conquer strategy. By a theoretical analysis, the algorithm is proved to be competitive with the state-of-the-art top-k algorithm on the compare time, with a significant improvement on the exchange time. On average, DC-Top-k takes at most (2-1/k)n+O(klog2k) comparisons and O(klog2k) exchanges to select the largest k terms from n terms. The effectiveness of the proposed algorithm is verified by a number of experiments which show that DC-Top-k is 1-3 times faster than Partial Quicksort and, moreover, is notably stabler than the latter. With an increase of k, it is also significantly more efficient than Min-heap based top-k algorithm (U. S. Patent, 2012). In the end, DC-Top-k is naturally implemented in a parallel computing environment, and a better scalability than Partial Quicksort is also demonstrated by experiments.

Summary Singular value decomposition (SVD) shows strong vitality in the area of information analysis and has significant application value in most of the scientific big data fields. However, with the rapid development of Internet, the information online reveals fast growing trend. For a large‐scale matrix, applying SVD computation directly is both time consuming and memory demanding. There are many works available to speed up the computation of SVD based on the message passing interface model. However, to deal with large‐scale data processing, a MapReduce model has many advantages over a message passing interface model, such as fault tolerance, load balancing and simplicity. For a MapReduce environment, existing approaches only focus on low rank SVD approximation and tall‐and‐skinny matrix SVD computation, and there are no implementations of full rank SVD computation. In this paper, we propose a MapReduce‐based implementation for solving divide‐and‐conquer SVD algorithm. To achieve high performance, we design a two‐stage task scheduling strategy based on the mathematical characteristics of divide‐and‐conquer SVD algorithm. To further strengthen the performance, we propose a row‐index‐based divide algorithm, a pipelined task scheduling method, and revised block matrix multiplication in MapReduce framework. Experimental result shows the efficiency of our algorithm. Our implementation can accommodate full rank SVD computation of large‐scale matrix very efficiently. Copyright © 2014 John Wiley & Sons, Ltd.

Smartphone contains diverse sensitive private information, such as phone number, contact and credit. The Android platform employs permission mechanism to protect intensive information from illegal leakage. There are several tools to assist users to manage apps' permissions, such as App Ops, Privacy Guard, and XPrivacy. However, the permission mechanism of Android is complicated, and only a few of smartphone users are familiar with Android security mechanism. In addition, it is difficult for smartphone users to know whether the privacy information exposure by Android applications is legal or not. Reverse engineering and program data flow analysis are basic approaches to analysis applications. Although it needs analysts to manual analysis results, its efficiency is largely depends on analysts' skills and experience. To improve analysis efficiency, we take a case-based reasoning method to simulation the security expert analysis applications. We translate the result of Android static analysis tools into cases and build a case reasoning library. Experiments show that it can help analysts to aid analysis Android applications.

With the rapid development of microblog, millions of Internet users share their opinions on different aspects of daily life. By analyzing and monitoring sentiment information extracting from tweets related to an important event, we are able to gain insights into variation trends of users’ sentiment. In this paper, we focus on extracting public sentiment of microblog emergencies. A subtopic-level opinion mining method is proposed based on two-phase optimization. Different subtopics of emergencies are extracted based on retweets. Opinion tweets are classified to different subtopics. The sentiment score of opinion holders is calculated. The above results are optimized based on users and endorsement interactions between users. Experimental results validate the effectiveness of the proposed method.

Summary Role mining was recently proposed to automatically find roles among user‐permission assignments using data mining technologies. However, the current studies about role mining mainly focus on how to find roles, without considering the constraints that are essentially required in role‐based access control systems. In this paper, we present a role mining algorithm with constraints, especially for the cardinality constraints. We illustrate it is essential for role mining to take cardinality constraints into account, and introduce the concepts of the cardinality constraints of roles and permissions. We further propose a role mining algorithm to generate roles based on these two kinds of cardinality constraints. The algorithm uses graph theory to model the role mining problem and maps the relation of two roles to the relation of graph elements. We set an optimization goal for role mining and employ graph optimization theory to find roles that satisfy the aforementioned cardinality constraints. We carry out the experiments to evaluate our approach. The experimental results demonstrate the rationality and effectiveness of the proposed algorithm. Copyright © 2015 John Wiley & Sons, Ltd.

Constraint is an essential aspect of role-based access control RBAC and is sometimes argued to be the principle motivation for RBAC. However, most of role mining algorithms do not consider the constraint. Furthermore, they just compare the least cost of the authorization process but do not consider how to assess the accuracy of the derived role state, thus, providing the motivation for this work. In this paper, we first define a wide variety of constraints, especially the permission cardinality constraint and user cardinality constraint. We further propose a role mining algorithm to generate roles based on these two kinds of cardinality constraints that consider the similarity between roles in the process of merging roles in order to improve the accuracy of the role state at the same time. Finally, we carry out the experiments to evaluate our approach. The experimental results demonstrate the effectiveness of our proposed algorithm. Copyright © 2014 John Wiley & Sons, Ltd.

By combining cloud computing and Peer-to-Peer computing, a P2P storage cloud can be formed to offer highly available storage services, lowering the economic cost by exploiting the storage space of participating users. However, since cloud severs and users are usually outside the trusted domain of data owners, P2P storage cloud brings forth new challenges for data security and access control when data owners store sensitive data for sharing in the trusted domain. Moreover, there are no mechanisms for access control in P2P storage cloud. To address this issue, we design a ciphertext-policy attribute-based encryption (ABE) scheme and a proxy re-encryption scheme. Based on them, we further propose a secure, efficient and fine-grained data Access Control mechanism for P2P storage Cloud named ACPC. We enforce access policies based on user attributes, and integrate P2P reputation system in ACPC. ACPC enables data owners to delegate most of the laborious user revocation tasks to cloud servers and reputable system peers. Our security analysis demonstrates that ACPC is provably secure. The performance evaluation shows that ACPC is highly efficient under practical settings, and it significantly reduces the computation overheads brought to data owners and cloud servers during user revocation, compared with other state-of-the-art revocable ABE schemes.

Topic modeling has become a widely used tool for document management. However, there are few topic models distinguishing the importance of documents on different topics. In this paper, we propose a framework LIMTopic to incorporate link based importance into topic modeling. To instantiate the framework, RankTopic and HITSTopic are proposed by incorporating topical pagerank and topical HITS into topic modeling respectively. Specifically, ranking methods are first used to compute the topical importance of documents. Then, a generalized relation is built between link importance and topic modeling. We empirically show that LIMTopic converges after a small number of iterations in most experimental settings. The necessity of incorporating link importance into topic modeling is justified based on KL-Divergences between topic distributions converted from topical link importance and those computed by basic topic models. To investigate the document network summarization performance of topic models, we propose a novel measure called log-likelihood of ranking-integrated document-word matrix. Extensive experimental results show that LIMTopic performs better than baseline models in generalization performance, document clustering and classification, topic interpretability and document network summarization performance. Moreover, RankTopic has comparable performance with relational topic model (RTM) and HITSTopic performs much better than baseline models in document clustering and classification.

Recently, there are many approaches proposed for mining roles using automated technologies. However, it lacks a tool set that can be used to aid the application of role mining approaches and update role states. In this demonstration, we introduce a tool set, RMiner, which is based on the core of WEKA, an open source data mining tool. RMiner implements most of the classic and latest role mining algorithms and provides interactive tools for administrator to update role states. The running examples of RMiner are presented to demonstrate the effectiveness of the tool set.

The community and the topic can help summarize the text-augmented social networks. Existing works mixed up the community and the topic by regarding them as the same. However, there is an inherent difference between the community and the topic such that considering them as the same is not so flexible. We propose a mutual enhanced infinite (MEI) community–topic model to detect communities and topics simultaneously in text-augmented social networks. The community and the topic are correlated via a community–topic distribution. The mutual enhancement effect between the community and the topic is validated by introducing two novel measures perplexity with community (perplexityc) and mean of the rank (MRK) with topic (MRKt). To determine the numbers of communities and topics automatically, the Dirichlet Process Mixture (DPM) model and the Hierarchical Dirichlet Process mixture (HDP) model are used to model the community and the topic, respectively. We further introduce parameters to model the weight of the community and the topic responsible for community inference. Experiments on the co-author network built from a subset of DBLP data show that MEI outperforms the baseline models in terms of the generalization performance. Parameter study shows that MEI is averagely improved by 3.7 and 15.5% in perplexityc and MRKt, respectively, by setting a low weight for the topic. We also experimentally validate that MEI can determine the appropriate numbers of communities and topics.

Topic modeling has become a widely used tool for document management due to its superior performance. However, there are few topic models distinguishing the importance of documents on different topics. In this paper, we investigate how to utilize the importance of documents to improve topic modeling and propose to incorporate link based ranking into topic modeling. Specifically, topical pagerank is used to compute the topic level ranking of documents, which indicates the importance of documents on different topics. By retreating the topical ranking of a document as the probability of the document involved in corresponding topic, a generalized relation is built between ranking and topic modeling. Based on the relation, a ranking based topic model Rank Topic is proposed. With Rank Topic, a mutual enhancement framework is established between ranking and topic modeling. Extensive experiments on paper citation data and Twitter data are conducted to compare the performance of Rank Topic with that of some state-of-the-art topic models. Experimental results show that Rank Topic performs much better than some baseline models and is comparable with the state-of-the-art link combined relational topic model (RTM) in generalization performance, document clustering and classification by setting a proper balancing parameter. It is also demonstrated in both quantitative and qualitative ways that topics detected by Rank Topic are more interpretable than those detected by some baseline models and still competitive with RTM.

Cloud computing is becoming increasingly prevalent in recent years. It introduces an efficient way to achieve management flexibility and economic savings for distributed applications. To take advantage of computing and storage resources offered by cloud service providers, data owners must outsource their data onto public cloud servers which are not within their trusted domains. Therefore, the data security and privacy become a big concern. To prevent information disclosure, sensitive data has to be encrypted before uploading onto the cloud servers. This makes plain text keyword queries impossible. As the total amount of data stored in public clouds accumulates exponentially, it is very challenging to support efficient keyword based queries and rank the matching results on encrypted data. Most current works only consider single keyword queries without appropriate ranking schemes. The multi-keyword query problem was being considered only recently. MRSE [1] is one of the first research works to define and address the problem of effective yet secure ranked multi-keyword search over encrypted cloud data. However, the keyword dictionary used in MRSE is static and must be rebuilt when the number of keywords in the dictionary increases. It also has severe out-of-order problems in the matching results and does not take the keyword access frequencies into account, which greatly affects its usability. In this paper, we propose a novel approach, called MKQE, to address these issues. Only minor changes in the dictionary structure have to be done when extra keywords are introduced. We also introduce new trapdoor generation and scoring algorithms to make in-order query results. Furthermore, the keyword access frequency is considered so as to select an adequate matching file set. We conduct extensive simulations and the results prove that our approach performs much better than previous solutions.

Large-scale search engines use hard disk drives (HDD) to store the mass index data for their capacity, whose performances are limited by the relatively low I/O performance of HDD. Caching is an effective optimization, and many caching algorithms have been proposed to improve retrieval performance. Considering the high cost of memory and huge amounts of data, the limited capacity of cache in memory cannot resolve the above problem thoroughly. In this paper, we adopt a solid state disk (SSD) based storage architecture, which uses SSD as a secondary cache for memory. We analyze the I/O patterns of search engines and propose SSD-based data management policies based on the hybrid storage architecture, including data selection, data placement and data replacement. Our main goal is to improve the performance of search engines while reducing operation cost inside SSD. The experimental results demonstrate the proposed architecture improves the hit ratio by 13.31%, the performance by 41.05%, the average access time inside SSD by 43.83%, and reduces block erasure operations by 71.52%.

NAND flash memory based Solid State Drive (SSD) is increasingly popular as one of the major non-volatile storage devices. Due to the superior performance and energy efficiency properties, it becomes an important complimentary device between the main memory and the traditional mechanical Hard Disk Drive (HDD). It is also anticipated to substitute HDD as the mainstream secondary storage. Today, flash memory is widely used in embedded systems, hand-held devices, personal computers and even enterprise computer systems. To access the data on the flash, a software component called Flash Translation Layer (FTL) has to be applied to convert the file system logical address into the corresponding physical address. FTL has great impacts on the system overall performance. Numerous FTL algorithms have been proposed in the past decade. DFTL is one of the most popular page-level address mapping FTL algorithms. It has been considered to have the best flexibility. However, it has extra mapping information I/O overhead and cannot always achieve the optimal performance. In this paper, we propose CAST, a novel and efficient pagelevel FTL algorithm to relieve this issue. CAST reserves a small portion of embedded SRAM to cache most recently accessed logical-physical address mapping information. Unlike DFTL, we use a compact packing methodology. Consecutive logical-physical page mapping information is represented with only a single entry. Thus, more address mapping information can be maintained in the caching table, and the cache hit rates can be increased. To improve the garbage collection efficiency, CAST maintains multiple current data blocks simultaneously. When a new data write request comes, the system can select an appropriate one to conduct the process based on the request issuer and/or logical address information. Our simulation results show that CAST outperforms DFTL under various workloads and it can reduce the number of erase operations and decrease the I/O response time significantly.

Recent advances in flash memory show great potential to replace traditional hard drives (HDDs) with flash-based solid state drives (SSDs) from personal computing to distributed systems. However, it is still a long way to go before completely using SSDs for enterprise data storage. Considering the cost, performance, and reliability of SSDs, a practical solution is to combine both SSDs and HDDs together. This paper proposes a hybrid storage system named PASS (Performance-dAta Synchronization - hybrid storage System) to tradeoff between I/O performance and data discrepancy between SSDs and HDDs. PASS includes a high-performance SSD and a traditional HDD to store mirrored data for reliability. All of the I/O requests are redirected to the primary SSD first and then the updated data blocks are copied to the backup HDD asynchronously. In order to hide the latency of copying operations, we use an I/O window to coalesce write requests and maintain an ordered I/O queue to shorten the HDD seek and rotation times. Depending on the charateristics of different I/O workloads, we develop an adaptive policy to dynamically balance the foreground I/O processing and background mirroring. We implement a prototype system of PASS by developing a Linux device driver and conduct experiments on the IoMeter, PostMark, and TPCC benchmarks. Our results show that PASS can achieve up to 12 times the performance of a RAID1 storage system for the IoMeter and PostMark workloads while tolerating less than 2% data discrepancy between the primary SSD and the backup HDD. More interestingly, while PASS does not produce any performance benefit for the TPC-C benchmark, it does allow the system to scale to larger sizes than when using an HDD-based RAID system alone.

The cipher text-based full-text retrieval technology is proposed to satisfy the pressing demands for high security in massive information sharing applications. Based on the analysis of general technology of full-text indexing, we propose an improved index structure that supports cipher text indexing and retrieval. Then we present approaches to construct, maintain and retrieve cipher text indexes. Meanwhile, we implement a cipher text-based full-text retrieval system, which allows full-text search on the encrypted documents in multiple formats without decryption. Finally, we carry out the experiments and evaluate the performance of cipher text index construction and retrieval.

Today, P2P system is one of the largest Internet bandwidth consumers. In order to relieve the burden on Internet backbone and improve the user access experience, efficient caching strategies should be applied. However, due to its autonomous nature, a fully distributed caching scheme is very difficult to design and implement. Most current P2P caching approaches are using Client/Server architecture by deploying dedicated proxy servers on the edge of networks. Such architecture is expensive. It also incurs single point of failure and hot spot problems. Furthermore, it violates P2P principle and failed to utilize vast available resources on individual peers. In this paper, we investigate the techniques for efficient distributed P2P caching. We propose novel placement and replacement algorithms to make caching decisions. For each object, an adequate number of copies are generated and disseminated on topologically distant locations. Combined with the underlying hierarchical query infrastructure, our strategies relieve the over-caching problems for popular objects, and provide more cache space for other objects. This resolution greatly reduces WAN traffic for P2P applications. We conduct simulation experiments to compare our approaches with several common caching strategies. The results show that our algorithms can achieve higher cache hit rates and superior load balance property.

As an important part of searching result presentation, query-biased document snippet generation has become a popular method of search engines that makes the result list more informative to users. Generating a single snippet is a lightweight task. However, it will be a heavy workload to generate multiple snippets of multiple documents as the search engines need to process large amount of queries per second, and each result list usually contains several snippets. To deal with this heavy workload, we propose a new high-performance snippet generation approach based on CPU-GPU hybrid system. Our main contribution of this paper is to present a parallel processing stream for large-scale snippet generation tasks using GPU. We adopt a sliding document segmentation method in our approach which costs more computing resources but can avoid the common defect that the high relevant fragment may be cut off. The experimental results show that our approach gains a speedup of nearly 6 times in average process time compared with the baseline approach-Highlighter.

Network coding has been demonstrated to be able to improve the performance of P2P content distribution. However, it is vulnerable to pollution attacks, leading to substantial performance degradation. Moreover, existing corruption detection schemes for network coding are not applied well to P2P systems. More efficient scheme based on attacker identification is required to thwart such attacks. In this paper, we propose an integrated system solution for secure P2P content distribution based on network coding, referred to as ISNC. In ISNC, we first design our system architecture based on extended uniform bipartite networks that can achieve high throughput with network coding. Based on the architecture, we present a secure network coding signature scheme and an identity-based malicious peer identification scheme. The two schemes can cooperate to thwart pollution attacks effectively in P2P network, not only detecting corrupted blocks, but also identifying all the malicious peers. Simulation results show that ISNC can effectively limit the pollution spread and identify malicious peers quickly, even when they collude to launch attacks. Compared with existing related schemes, ISNC is especially applicable for P2P content distribution, and can achieve both high security and overall efficiency.

Cloud-oriented service mashup can aggregate many services to provide personalized services for end-users on demand. However, how to securely aggregate mashup services becomes a bottleneck of hampering the development of cloud computing. In this paper, we present a secure cloud service mashup framework called SMEF to address this problem. In SMEF, we employ security entropy to measure the unascertained security degree of service mashup. The nonfunctional criteria of SMEF are aggregated as a single criterion by defining a utility function. Then the relatively optimal mashup services are selected to meet the user requirements. Finally, we have implemented a simulation of SMEF and conducted extensive experiments using simulations of different sizes of services and security factors. Experimental results show the feasibility and efficiency of the SMEF service mashup framework.

Abstract The principle of least privilege in role‐based access control is an important area of research. There are two crucial issues related to it: the specification and the enforcement. We believe that the existing least privilege specification schemes are not comprehensive enough and few of the enforcement methods are likely to scale well. In this paper, we formally define the basic principle of least privilege problem and present different variations, called the delta‐approx principle of least privilege problem and the minimizing‐approx principle of least privilege problem. Since there may be more than one result to enforce the same principle of least privilege, we introduce the notation about weights of permissions and roles to optimize the results. Then we prove that all least privilege problems are NP‐complete. As an important contribution of the paper, we show that the principle of least privilege problem can be reduced to minimal cost set covering (MCSC) problem. We can borrow the existing solutions of MCSC to solve the principle of least privilege problems. Finally, different algorithms are designed to solve the proposed least privilege problems. Experiments on performance study prove the superiority of our algorithms. Copyright © 2011 John Wiley & Sons, Ltd.

In distributed environments, statements from a number of principals, besides the central trusted party, may influence the derivations of authorization decisions. However, existing authorization logics put few emphasis on this set of principals - authorization provenance. Reasoning about provenance enables to (1) defend against a class of attacks, (2) understand and analyze authorizations and the status of policy bases, and (3) obtain potentially efficient logging and auditing guided by provenance information. This paper presents the design and applications of a provenance-enabled authorization logic, called DBT. More specifically, we give a sound and complete axiomatic system of DBT. We also examine a class of provenance-aware policy bases and queries. One can syntactically extract provenance information from the structure of these queries if they are evaluated positively in provenance-aware policy bases. Finally, two case studies are presented to demonstrate possible applications of DBT.

Traditional web application development often encounters tight coupling problem between access control logic and business logic. It is hard to configure and modify access control policies after a system has been deployed. In this demonstration, we present SecTag, a multi-policy supported secure web tag framework, to address this problem. We define a series of general-purpose secure attributes that meet the demand of fine-grained access control in web presentation layer. We also design a set of high interactive secure tags, which encapsulate secure features to provide reusable secure components for web development. A running example of SecTag is presented to demonstrate the effectiveness of the proposed framework.

In order to access sensitive documents shared over government, army and enterprise intranets, users rely on an indexing facility where they can quickly locate relevant documents they are allowed to access, (1) without leaking information about the remaining documents, (2) without imposing large load

Social tagging is an increasingly popular way to describe and classify documents on the web. However, the quality of the tags varies considerably since the tags are authored freely. How to rate the tags becomes an important issue. In this paper, we propose a topic-sensitive tag ranking (TSTR) approach to rate the tags on the web. We employ a generative probabilistic model to associate each tag with a distribution of topics. Then we construct a tag graph according to the co-tag relationships and perform a topic-level random walk over the graph to suggest a ranking score for each tag at different topics. Experimental results validate the effectiveness of the proposed tag ranking approach.

The role-based access control (RBAC) has significantly simplified the management of users and permissions in computing systems. In dynamic environments, systems are usually undergoing changes, whereas the associated user-role, role-role and role-permission relations need to be updated accordingly in order to reflect the systems’ evolutions. However, such updating process is generally complicated as the resulting system state is expected to meet necessary constraints. This paper presents an approach for assisting administrators with the update task: using this approach, it is possible to check, in an automatic way, whether a required update is achievable or not, and if so, a reference model will be produced. In light of this model, administrators could fulfill the changes to RBAC systems. We propose a formalization of the update approach, investigate its properties, and develop an updating algorithm based on model checking techniques. Our experimental results demonstrate the effectiveness of our approach.

Role mining from the existing permissions has been widely applied to aid the process of migrating to an RBAC system. While all permissions are treated evenly in previous approaches, none of the work has employed the weights of permissions in role mining to our knowledge, thus providing the motivation for this work. In this paper, we generalize this to the case where permissions are given weights to reflect their importance to the system. The weights can correspond to the property of operations, the sensitive degree of objects, and the attribute of users associated with permissions. To calculate the weight of permissions, we introduce the concept of similarity between both users and permissions, and use a similarity matrix to reinforce the similarity between permissions. Then we create a link between the reinforced similarity and the weight of permissions. We further propose a weighted role mining algorithm to generate roles based on weights. Experiments on performance study prove the superiority of the new algorithm.

The blind flooding algorithm under a time-to-live (TTL) constraint used in unstructured peer-to-peer (P2P) networks, such as Gnutella, may cause a large amount of network traffic. The algorithm cannot guarantee for acquiring the required data objects, especially for rare ones. To mitigate these problems, this paper proposes PercolationNET, a multi-tree sub-overlay, which is built on top of an existing P2P overlay (named original overlay). PercolationNET organizes peers in a tree-based structure which facilitates reliable and efficient message dissemination for search. The search process is divided into two stages. A query message is first propagated on the original overlay, and then broadcast along the sub-overlay PercolationNET. PercolationNET combines the advantages of fast coverage speed in flooding-based scheme and low traffic cost in tree-based scheme. The experimental results of PercolationNET compared with FloodNet confirm the superiority of PercolationNET in achieving faster coverage speed and lower message cost.

In distributed environments, access control decisions depend on statements of multiple agents rather than only one central trusted party. However, existing policy languages put few emphasis on authorization provenances. The capability of managing these provenances is important and useful in various security areas such as computer auditing and safeguarding delegations. Based on the newly proposed logic, we define one type of authorization provenances. We exemplify the applications of these provenances by a case study.

The inter-domain role mapping is a basic method for facilitating interoperation in RBAC-based collaborating environments, where each domain employs role based access control (RBAC) to specify access control policies. Prior to concrete interoperation, one important problem is to establish role mappings. Two issues are involved in the establishing process. The first one is to generate role mappings while respecting RBAC states such as separation of duty (SoD) constraints. On the other hand, administrative works of RBAC policies are sometimes needed to generate mappings. This paper investigates these two problems, mostly from the computational perspective. In particular, we study how to find a set of roles appropriate for mappings and how to fulfill interoperation requests; it turns out that most of corresponding problems are NP-complete. Further, several useful subcases of these problems are identified. We also motivate and support partial interoperation by imposing constraints on interoperation requests. When administrative works are necessary, we examine how to minimize administrative cost; the result is that one subcase of the problem reduces to the "minimal set cover" (MSC) problem. We demonstrate that approaches to MSC can be applied to this problem, even though they are not totally equivalent. Finally, a discussion on how administrative operations made to RBAC states may influence interoperability is presented as well.

Peer-to-peer (P2P) technology provides an efficient way for resource distribution, and sharing. While most current P2P systems only support queries over a single attribute which limits the popularity of the P2P technology. Full-blown P2P applications require the efficient resource searching supporting multi-dimensional attributes. In this paper, we propose flabellate overlay network (FAN), a scalable P2P overlay supporting multi-dimensional attributes. In FAN, the peers are mapped into a d-dimensional Cartesian space. The resource management and searching are based on the peer's second moment to the origin of coordinates. The theoretical analyses and experimental results demonstrate that FAN has high routing efficiency and low network maintenance cost over the existing structured P2P systems by storing logarithmic routing messages in peers and achieving logarithmic-hop resource searching. And many improved routing algorithms supporting multi-attribute queries can be implemented over FAN and achieve better performance.

Real-time collaborative editing systems allow a group of users to view and edit the same document at the same time from geographically dispersed sites connected by communication networks. Consistency maintenance in concurrent access of shared objects is one of the key issues in designing these types of systems. This paper proposes a P2P-based locking schema for concurrency control in internet-based real-time collaborative editing systems. This schema can distribute locking services to all collaborative sites and quickly find out the locking service nodes of given shared objects. The characteristic of distributed locking services solves the hot spot problem in centralized locking schema, and the capability of quickly locating locking service nodes resolves the broadcast communication problem in the existing distributed locking schemas. It also provides the basic locking function in collaborative editing systems and can be used in varied locking polices.

This paper designs an efficient cooperative caching architecture for images, named iCache, which introduces super cache servers in caching system and caches images using different policies according to image size. The small-size images are stored in cache server directly, while the large-size images are divided into small blocks which are distributed in multiple cache servers in order to achieve scalability and load balance. Furthermore, the paper proposes corresponding storage policy and replacement mechanism for iCache that stores the least copies of large-size images to save space and giving them higher priority to keep in cache than small ones for increasing the byte hit rate. The simulation experiments are carried out to evaluate the performance of iCache architecture and the results show that the proposed scheme offers good search efficiency and hit rate with cooperation overhead significatively lower than that of other cooperative mechanisms.

The problem is that the parallel scheduling strategies are minimally supported in distributed environments. The well-known Coffman-Graham algorithm is highly effective but applied only for the sample case when all tasks have the same execution time and not applied for the distributed systems. In this paper, an expanding task graph and an expanding Coffman-Graham algorithm, which orients to the distributed systems and can be applied for the case when the tasks have the different execution time, are proposed. Like the well-known Coffman-Graham algorithm, the ECG algorithm is proved to be an efficient scheduling algorithm for the distributed systems, through presenting and deducting one theorem and two corollaries..

Despite the recent advances in access control approaches applicable to grid computing, there remain issues that impede the development of effective access control for grid applications. Amongst them are the lack of context-based models for access control, and reliance on identity or capability-based access control schemes. In this paper, we propose RCBAC model which extends the RBAC with context constraints. The RCBAC mechanisms dynamically grant and adapt permissions to users based on a set of contextual information collected from the grid environments, while retaining the advantages of RBAC model.