Despite the rapid progress in large language models (LLMs), even sub-billion-scale systems perform at chance level on challenging natural language inference (NLI) benchmarks such as Adversarial Natural Language Inference (ANLI), while training larger models is often impractical due to limited computational resources. We address this parameter-efficiency bottleneck in NLI with a Complex-Vector Token Representation that explicitly decouples each token from its context, and a Token-Context Attention mechanism that updates each token based on the most informative contextual semantics. On ANLI, a 0.8B-parameter Token-Context Attention model achieves higher parameter efficiency (accuracy per parameter) than all 1B and comparable 0.8B self-attention baselines; it also suffers smaller performance degradation under Fast Gradient Sign Method (FGSM) and Projected Gradient Descent (PGD) attacks and achieves the largest few-shot gains on SNLI and MNLI while exhibiting no significant degradation in ANLI accuracy after adaptation. These results suggest that explicitly disentangling token and context offers a viable alternative to standard self-attention for NLI tasks.

Inductive knowledge graph completion (KGC) aims to represent unseen entities and complete triplets in emerging knowledge graphs (KGs), while the existing studies ignore that unseen elements combined with seen ones constitute a holistic new relational graph, where emerging KGs have inescapable impacts backtracking to original ones. Therefore, it is not only necessary to predict triplets in emerging KGs, but also with particular significance to further improve the completeness of original ones, considering the semantic and topological variations in the holistic new graph. To fill in this gap, we formulate a newIT(Inductive-Transductive) KGC task to transductively complete triplets inside original KGs after entities in the emerging scenario are represented and fine-tuned in an inductive manner. In order to handle this task, a novel model entitled StaR (Self-adaptive Retroaction-awareRepresentation) is proposed consisting of the following two modules: 1) a self-adaptive semantic encoding network is designed to adaptively adjust embeddings of seen entities to their surrounding semantic mutations; 2) a relation-aware transformer layer is developed to represent both seen and unseen entities in a unified representation space and generalize evolving reasoning paradigms to the whole graph. Our experimental results demonstrate that, compared with state-of-the-art methods, StaR is not only competitive in inductive KGC for unseen entities, but also ulteriorly improves the completeness of original parts inside the holistic new relational graph in ourITKGC task.

Fusing visible (RGB) and thermal (T) images for RGBT tracking has received growing interest in the field of computer vision. However, how to improve the robustness of the tracker to target scale variety, effectively apply visual prompts to multimodal tracking tasks, and enhance the multimodal fusion effectiveness are still urgent challenges in the field of RGBT tracking. To this purpose, this work proposes an RGBT tracking framework integrating scale-aware dilation attention, multimodal prompt interaction learning, and cross- fusion adapter, named MPANet. Firstly, a scale-aware dilation attention (SADA) module is put forward to enhance the flexibility of the tracker in the presence of target scale variations by embedding convolutions with different dilation rates into the self-attention. Subsequently, a multimodal prompt interaction learning (MPIL) module is constructed, which combines global token adaptive attention and spatial attention to efficiently learn visual prompts from different modalities and achieve intermodal prompt interactions. Finally, a cross-fusion adapter (CFA) is developed to facilitate the adaptability of the network to different modalities in the process of multimodal information fusion through the adapter mechanism. Extensive experiments on public RGBT benchmark tracking datasets such as GTOT, RGBT234, LasHeR and VTUAV demonstrate that the proposed method outperforms existing advanced trackers and achieves state-of-the-art performance.

Deep Reinforcement Learning (DRL) has garnered significant attention as a promising approach for developing intelligent and adaptive recommender systems. This paradigm is especially well-suited for recommendation scenarios characterized by dynamic user environments, temporally delayed feedback signals, and continuously shifting user preferences. However, deploying DRL within recommendation scenarios introduces a range of intricate challenges. These include the design of meaningful and task-aligned reward functions, effective navigation of vast and complex action spaces, and the need to maintain sample efficiency in data-sparse environments. Ensuring robust and stable training dynamics adds further difficulty. This special issue brings together a diverse collection of cutting-edge research that addresses these pressing challenges, showcasing advances that move the field toward more adaptive, robust, and personalized recommendation systems grounded in reinforcement learning.

Reentrancy vulnerabilities in smart contracts have been exploited to steal enormous amounts of money, thus detecting reentrancy vulnerabilities is a hotspot issue in security research. However, a new attack is emerging in which attackers continuously release new reentrancy patterns to exploit fresh vulnerabilities and obfuscate existing ones. Existing detection methods neglect the time-series evolution of vulnerabilities across different smart contract versions, leading to a gradual decline in their effectiveness over time. We investigate the time-series correlations among vulnerabilities in various versions and refer to these as Evolutionary Reentrancy Vulnerabilities (ERVs). We summarize that ERVs detection faces two key challenges: (i) capturing the evolving pattern of ERVs along a complete evolutionary chain and (ii) detecting fresh reentrancy vulnerabilities in new versions. To address these challenges, we propose CLEP, a novel Contrastive Learning with Evolving Pairs detection method. It can effectively capture the evolving patterns by discerning similarities and differences across versions. Specifically, we first modified the sample distribution by incorporating version declarations as time-series evolution information. Then, leveraging the hierarchical similarity, we design an evolving pairs scheme to form negative and positive contract pairs across versions. Finally, we build a complete evolutionary chain by proposing a version-aware contrastive sampler. Our experimental results show that CLEP not only outperforms state-of-the-art baselines in version-specific scenarios but also shows promising performance in cross-version evolution scenarios.

Most existing semi-supervised community detection algorithms leverage known communities to learn community structures, subsequently identifying communities that align with these learned community structures. However, differences in community structures may render the community structures learned by these methods inappropriate for the community containing the given node of interest. As a result, the identified community may exclude the given node or be of poor quality. Inspired by the success of reinforcement learning, we propose a Semi-supervised Local community detection method based on Reinforcement Learning, named SLRL, which only explores parts of the network surrounding the given node. It first extracts the local structure around a given node with an extractor, followed by selecting communities that are similar to this local structure to distill useful communities. These selected communities are employed to train the expander, which expands the community containing a given node. Experimental results demonstrate that SLRL outperforms state-of-the-art algorithms on five real-world datasets.

Collaborative reasoning enhances recommendation performance by combining the strengths of symbolic learning and deep neural learning. However, current collaborative reasoning models rely on parameterized networks to simulate logical operations within the reasoning process, which (1) do not comply with all axiomatic principles of classical logic and (2) limit the model's generalizability. To address these limitations, a Fuzzy logic approach tailored for Collaborative Reasoning (FuzzCR) is proposed in this work, aiming to augment the recommendation system with cognitive abilities. Specifically, this method redefines the sequential recommendation task as a logical query answering process to facilitate a more structured and logical progression of reasoning. Moreover, learning-free fuzzy logical operations are implemented for the designed reasoning process. Taking advantage of the inherent properties of fuzzy logic, these logical operations satisfy fundamental logical rules and ensure complete reasoning. After training, these operations can be applied to flexible reasoning processes, rather than being confined to fixed computation graphs, thereby exhibiting good generalizability. Extensive experiments conducted on publicly available datasets demonstrate the superiority of this method in solving the sequential recommendation task.

Protein-protein interaction (PPI) prediction is critical for understanding cellular mechanisms and accelerating drug discovery. Although recent deep learning methods leverage protein sequence and structural data, they often fail to capture long-range dependencies and complex interaction dynamics. We propose ComplexWaveGAT, a deep learning framework that uses complex-valued graph attention to model both magnitude and phase features inspired by wave-like biological interactions. The architecture combines multi-head complex attention with a WavePropagation module to simulate constructive and destructive interference, using hybrid protein graphs built from pretrained language model embeddings and secondary structure features. We evaluate the model on the Struct2Graph dataset using both standard cross-validation and strict protein-disjoint splits to assess generalization. Results show that ComplexWaveGAT outperforms sequence-based, structure-based, and real-valued graph baselines, demonstrating strong predictive capability and promising potential for biomedical applications.

Transcription factors (TFs) play a pivotal role in regulating gene expression, making accurate TF prediction critical for elucidating gene regulatory mechanisms. However, existing approaches face two key limitations: (1) traditional methods struggle with low accuracy, and (2) deep learning models require large datasets yet often lack interpretability, especially when domain-specific biological knowledge is underutilized. To address these challenges, we present LTN-TFpredict, a neurosymbolic framework that integrates Logic Tensor Networks (LTNs) with deep learning to simultaneously enhance accuracy and interpretability. Our model leverages pre-trained protein language models for high-dimensional sequence embeddings and incorporates logical constraints derived from five key TF-related motifs: zinc fingers, leucine zippers, basic helix-loop-helix, forkhead, and winged helix-turn-helix, to guide learning with biologically meaningful rules. This integration enforces consistency with known TF characteristics, improving both predictive performance and biological validity. Experimental results demonstrate that LTN-TFpredict consistently outperforms traditional classifiers (TFpredict, XGBoost), CNN-based models (DeepTFactor, ProtCNN), transformer-based architectures (ESM-TFpredict, ProtT5), and two alternative strategies: multi-loss optimization and motif-based data augmentation, achieving state-of-the-art accuracy while preserving strict logical compliance with domain knowledge. By bridging deep learning and symbolic reasoning, LTN-TFpredict offers a robust, interpretable, and biologically grounded solution for TF prediction, advancing the application of neurosymbolic AI in computational biology.

A fundamental challenge in remote sensing-based forest fire detection lies in accurately discerning fire characteristics on various scales against the backdrop of intricate and heterogeneous forest landscapes. In response to this challenge, we propose a dual-path network (DPMNet) with multidimensional feature interaction for real time remote sensing forest fire detection. Initially, a dual-path backbone network is designed, integrating coarse-grained and fine-grained parallel pathways, working in tandem to capture both global visual features and nuanced local texture details. Subsequently, we develop the Multidimensional Interactive Feature Pyramid Network (MiFPN), a novel structure that amalgamates information streams from varied levels through a three-branch structure and engenders profound fusion and dynamic interaction of features across multiple scales. Thereafter, the Context-Enriched Adaptive Fusion Module (CEAFM) is proposed, which emerges to meticulously blend macroscopic visual elements harvested via coarse-grained conduits, employing a multi-faceted pathway strategy to bolster the model’s overarching comprehension and precision in forest fire detection. Finally, the Enhanced Contextual Pooling Bottleneck (ECPB) is put forward, an integration that augments the model’s spatial perception and contextual acumen through the incorporation of dilated convolution and global pooling techniques. Extensive experiments are conducted on the remote sensing forest fire dataset in order to confirm the efficacy of DPMNet. The experimental results demonstrate that our DPMNet achieves satisfactory performance in terms of real-time performance as well as accuracy and provides an effective solution for real-time detection of remote sensing forest fires based on UAVs.

Recently masked autoencoder (MAE) has achieved great success in visual representation learning and delivered promising potential in many downstream vision tasks. However, due to the lack of saliency supervision signal in original MAE, almost no saliency information can be learned from masked image reconstruction process. Therefore, salient object detection (SOD) can hardly benefit from MAE pretraining. To address this issue, we integrate SOD model and saliency supervision in MAE and propose a simple and effective framework DHMMAE utilizing MAE with dynamic hybrid masking ratios to pretrain SOD model. Specifically, we treat MAE as an online data augmenter to generate endless pseudo images for SOD model to predict saliency maps, where saliency supervision is employed to ensure the model can learn robust saliency prior knowledge from the reconstructed images. Besides, we propose a simple and novel network EANet driven by DHMMAE pretraining for SOD in optical remote sensing images (ORSIs). Two key modules are designed for EANet to further enhance the performance: enhanced diverse feature aggregation module (EDFAM) and adjacent-context shuffle spatial attention module (ASSAM). EDFAM aggregates diverse features via three different types of convolution layer and enhances them by convolution block attention module. ASSAM captures spatial location information of salient objects by employing channel shuffle operation and weighted spatial attention mechanism on the fused adjacent context. Experiments on three ORSI-SOD datasets demonstrate that our proposed method outperforms the cutting-edge methods. Code is available at: https://github.com/Voruarn/EANet.

The information available in multi-attributed road-social networks includes network structure, location information, and numerical attributes. Most studies mainly focus on mining communities by combining structure with attributes or structure with location, which do not consider structure, attributes, and location simultaneously. Therefore, we propose a parameter-free algorithm, called LCDMRS, to mine local communities in multi-attributed road-social networks. LCDMRS extracts a sub-network surrounding the given node and embeds it to generate the vector representations of nodes, which incorporates both structural and attributed information. Based on the vector representations of nodes, the average cosine similarity between nodes is designed to ensure both the structural and attributed cohesiveness of the community, while the community node density is designed to ensure the spatial cohesiveness of the community. Targeting the community node density and cosine similarity of nodes, LCDMRS takes the given node as the starting node and employs the community dominance relation to expand the community outward. Experimental results on multiple real-world datasets demonstrate LCDMRS outperforms comparison algorithms.

Nowadays, detecting sentiment or emotion from user generated texts has been intensively studied in natural language understanding, especially via neural-based models based on text representation. However, the interpretability on how could the final text sentiment be determined by neural-based text representation has not been thoroughly unfolded yet. Consequently, in this paper, we proposeCogLignwhich injects theneural-cognitionderived from Electroencephalogram (EEG)-signal into theneural-basedtext sentiment analysis model, aimed at learning the activation of brain regions stimulated by different sentiments, so as to guide our proposedCogLignto make proper determination on text sentiment in brain-like way. Specifically, on the one hand, the given videos in different sentiments have been watched bysubjects, during which the EEG-signals are monitored to construct brain connectivity pattern asbrain graph(BG), attaining more obvious sentiment response on brain region activation forneural-cognition. On the other hand, we interpret the video-plots (or video-semantics) along timeline into text, where the entire video-interpreted-text will bestrictly boundwith the wholeEEG-signal-sequencebysegmentvia the fixed size oftime-window. Then, entities and relations are extracted from the video-interpreted-text to constructknowledge graph(KG), depicting text semantics. Next, mapping fromentities(or nodes) inKGtoEEG-Electrodes(or nodes) inBG, further dated back to different brain regions, has been learned viacognition alignmentbetween the EEG-derivedBGand text-derivedKG. In this way, by aligningneural cognitionfrombrain graphwith thesemantic cognitionfromknowledge graph, our proposed frameworkCogLigncan not only achieve the overall best sentiment analysis performance on thevideo-interpreted-text, but can also detect brain connectivity patterns in different sentiments more consistent with the prior conclusion of brain region sentiment preference, revealing competitiveinterpretabilityon text sentiment determination.

The success of deep learning in 3D medical image segmentation hinges on training with a large dataset of fully annotated 3D volumes, which are difficult and time-consuming to acquire. Although recent foundation models (e.g., segment anything model, SAM) can utilize sparse annotations to reduce annotation costs, segmentation tasks involving organs and tissues with blurred boundaries remain challenging. To address this issue, we propose a region uncertainty estimation framework for Computed Tomography (CT) image segmentation using noisy labels. Specifically, we propose a sample-stratified training strategy that stratifies samples according to their varying quality labels, prioritizing confident and fine-grained information at each training stage. This sample-to-voxel level processing enables more reliable supervision information to propagate to noisy label data, thus effectively mitigating the impact of noisy annotations. Moreover, we further design a boundary-guided regional uncertainty estimation module that adapts sample hierarchical training to assist in evaluating sample confidence. Experiments conducted across multiple CT datasets demonstrate the superiority of our proposed method over several competitive approaches under various noise conditions. Our proposed reliable label propagation strategy not only significantly reduces the cost of medical image annotation and robust model training but also improves the segmentation performance in scenarios with imperfect annotations, thus paving the way towards the application of medical segmentation foundation models under low-resource and remote scenarios. Code will be available at https://github.com/KHan-UJS/NoisyLabel.

The tremendous positive driving effect of Graph Convolutional Network (GCN) and Graph Contrastive Learning (GCL) for recommender systems has become a consensus. GCN encoders are extensively used in recommendation models for capturing high-order connectivities between users and items, whereas GCL accelerates the training of recommendation tasks by adding extra supervision signals from contrastive objectives. However, little attention has been paid on corresponding theories that are truly tailored to recommendation tasks. From the technical perspective, Collaborative Filtering (CF) is seen as an important factor in recommender systems. It is applied to measure user–user, item–item, and user–item similarities rather than to achieve better clustering or node classification results. Besides, heuristic-based data augmentation may not be hold true in the field of recommender systems as it requires additional training costs and introduces noises that will corrupt the interaction graph structure and the semantic information of nodes. To tackle these limitations, we propose a novel Embedding-Less Graph Collaborative Filtering (EGCF) for recommendation, which is tailor-made for the problem mentioned for CF and further simplifies existing solutions. Structurally, it consists of two parts: embedding-less GCN and embedding-less GCL. The former improves user–item affinity by streamlining user-type embeddings and carrying out iterative graph convolution. And the latter utilizes three-type contrastive objectives to directly measure the alignment and the uniformity of users, items, and interaction pairs, respectively, avoiding any type of data augmentation or multi-view construction. Even though EGCF has been extremely streamlined, extensive experimental results on three classical datasets demonstrate the effectiveness of EGCF in terms of recommendation accuracy and training efficiency. The code and used datasets are released at https://github.com/BlueGhostYi/ID-GRec .

Abstract Quantum machine learning as a field has emerged rather quickly thanks to developments in quantum computing. Of these developments, the one of primary interest is the Quantum Generative Adversarial Network or QGAN which is an enhancement of the familiar GAN to use quantum computation necessary for producing synthetic images. To sum up, based on different types of experiments, QGANs outperformed classical GANs, especially in cases with images such as MNIST and Fashion MNIST datasets. Nevertheless, their capabilities are not fully comprehensible due to existing constraints in quantum systems technology, especially in the NISQ era. In this regard, the current study undertakes a proposed research direction that focuses on improving the resolution of grayscale images that have been produced from the “optdigits” dataset, which contains handwritten digit images. Our work then contrasts this with prior work in terms of FID scores, loss function values, runtime, and the resolution of the images. Further, we extend the work by carrying out the proposed methodology on the FMNIST dataset and provide results to corroborate the efficacy of the proposed technique, besides enabling comparison on the same platform with prior works.

Transcription factors (TFs) play a fundamental role in gene regulation by selectively binding to specific DNA sequences. Understanding the nature and behavior of these TFs is essential for insights into gene regulation dynamics. In this study, we introduce a robust multi-task learning framework specifically tailored to harness both TF-specific annotations and TF-related domain annotations, thereby enhancing the accuracy of TF predictions. Notably, we incorporate cutting-edge language models that have recently garnered attention for their outstanding performance across various fields, particularly in biological computations like protein sequence modeling. Comparative experimental analysis with existing models, DeepTFactor and TFpredict, reveals that our multi-task learning framework achieves an accuracy exceeding 92% across four evaluation metrics on the TF prediction task, surpassing both competitors. Our work marks a significant leap in the domain of TF prediction, enriching our comprehension of gene regulatory mechanisms and paving the way for the discovery of novel regulatory motifs.

With the help of Vision transformers (ViTs), medical image segmentation was able to achieve outstanding performance. In particular, they overcome the limitation of convolutional neural networks (CNNs) which rely on local receptive fields. ViTs use self-attention mechanisms to consider relationships between all image pixels or patches simultaneously. However, they require large datasets for training and did not perform well on capturing low-level features. To that end, we propose DDViT, a novel ViT model that unites a CNN to alleviate data-hunger for medical image segmentation with two multi-scale feature representations. Significantly, our approach incorporates a ViT with a plug-in domain adapter (DA) with Double-Level Fusion (DLF) technique, complemented by a mutual knowledge distillation paradigm, facilitating the seamless exchange of knowledge between a universal network and specialized domain-specific network branches. The DLF framework plays a pivotal role in our encoder-decoder architecture, combining the innovation of the TransFuse module with a robust CNN-based encoder. Extensive experimentation across diverse medical image segmentation datasets underscores the remarkable efficacy of DDViT when compared to alternative approaches based on CNNs and Transformer-based models.

Large language models like ChatGPT can generate human-like code, posing challenges for programming education as students may be tempted to misuse them on assignments. However, there are currently no robust detectors designed specifically to identify AI-generated code. This is an issue that needs to be addressed to maintain academic integrity while allowing proper utilization of language models. Previous work has explored different approaches to detect AI-generated text, including watermarks, feature analysis, and fine-tuning language models. In this paper, we address the challenge of determining whether a student's code assignment was generated by a language model. First, our proposed method identifies AI-generated code by leveraging targeted masking perturbation paired with comperhesive scoring. Rather than applying a random mask, areas of the code with higher perplexity are more intensely masked. Second, we utilize a fine-tuned CodeBERT to fill in the masked portions, producing subtle modified samples. Then, we integrate the overall perplexity, variation of code line perplexity, and burstiness into a unified score. In this scoring scheme, a higher rank for the original code suggests it's more likely to be AI-generated. This approach stems from the observation that AI-generated codes typically have lower perplexity. Therefore, perturbations often exert minimal influence on them. Conversely, sections of human-composed codes that the model struggles to understand can see their perplexity reduced by such perturbations. Our method outperforms current open-source and commercial text detectors. Specifically, it improves detection of code submissions generated by OpenAI's text-davinci-003, raising average AUC from 0.56 (GPTZero baseline) to 0.87 for our detector.

Reverse engineering involves analyzing the design, architecture, and functionality of systems, and is crucial for legacy systems. Legacy systems are outdated software systems that are still in use and often lack proper documentation, which makes their maintenance and evolution challenging. To address this, we introduce SC2Req, utilizing the Generative Pre-trained Transformer (GPT) for automated code analysis and requirement generation. This approach aims to convert source code into understandable requirements and bridge the gap between those two. Through experiments on diverse software projects, SC2Req shows the potential to enhance the accuracy and efficiency of the translation process. This approach not only facilitates faster software development and easier maintenance of legacy systems but also lays a strong foundation for future research, promoting better understanding and communication in software development.

In the era of large language models like Chatgpt, maintaining academic integrity in programming education has become challenging due to potential misuse. There's a pressing need for reliable detectors to identify Chatgpt-generated code. While previous studies have tackled model-generated text detection, identifying such code remains uncharted territory. In this paper, we introduce a novel method to discern Chatgpt-generated code. We employ targeted masking perturbation, emphasizing code sections with high perplexity. Fine-tuned CodeBERT is utilized to replace these masked sections, generating subtly perturbed samples. Our scoring system amalgamates overall perplexity, variations in code line perplexity, and burstiness. In this scoring scheme, a higher rank for the original code suggests it's more likely to be chatgpt-generated. The underlying principle is that code generated by models typically exhibits consistent, low perplexity and reduced burstiness, with its ranking remaining relatively stable even after subtle modifications. In contrast, human-written code, when perturbed, is more likely to produce samples that the model prefers. Our approach significantly outperforms current detectors, especially against OpenAI's text-davinci-003 model, with the average AUC rising from 0.56 (GPTZero baseline) to 0.87.

Recommender systems play a crucial role in delivering personalized services to users, but the increasing volume of user data raises significant concerns about privacy, security, and utility. However, existing machine unlearning methods cannot be directly applied to recommendation systems as they overlook the collaborative information shared across users and items. More recently, a method known as RecEraser was introduced, offering partitioning and aggregation-based approaches. Nevertheless, these approaches have limitations due to their inadequate handling of additional overhead costs. In this paper, we propose A General Strategy Graph Collaborative Filtering for Recommendation Unlearning (GSGCF-RU), which is a novel model-agnostic learnable delete operator that optimizes unlearning edge consistency and feature representation consistency. Specifically, the GSGCF-RU model utilizes unlearning edge consistency to eliminate the influence of deleted elements, followed by feature representation consistency to retain knowledge after deletion. Lastly, experimental results on three real-world public benchmarks demonstrate that GSGCF-RU not only achieves efficient recommendation unlearning but also surpasses state-of-the-art methods in terms of model utility. The source code can be found at https://github.com/YongjingHao/GSGCF-RU.

Sequential Recommendation (SR) has received increasing attention due to its ability to capture user dynamic preferences. Recently, Contrastive Learning (CL) provides an effective approach for sequential recommendation by learning invariance from different views of an input. However, most existing data or model augmentation methods may destroy semantic sequential interaction characteristics and often rely on the hand-crafted property of their contrastive view-generation strategies. In this paper, we propose a Meta-optimized Seq2Seq Generator and Contrastive Learning (Meta-SGCL) for sequential recommendation, which applies the meta-optimized two-step training strategy to adaptive generate contrastive views. Specifically, Meta-SGCL first introduces a simple yet effective augmentation method called Sequence-to-Sequence (Seq2Seq) generator, which treats the Variational AutoEncoders (VAE) as the view generator and can constitute contrastive views while preserving the original sequence's semantics. Next, the model employs a meta-optimized two-step training strategy, which aims to adaptively generate contrastive views without relying on manually designed view-generation techniques. Finally, we evaluate our proposed method Meta-SGCL using three public real-world datasets. Compared with the state-of-the-art methods, our experimental results demonstrate the effectiveness of our model and the code is available.11https.//anonymous.4open.science/status/Meta-SGCL-05B5

Graph Collaborative Filtering (GCF) is designed to leverage high-order connectivity in user-item graphs, thereby significantly enhancing recommendation performance. Recent advancements have seen the integration of contrastive learning into GCF as a strategy to mitigate the challenges of data sparsity. This approach involves creating contrastive views through augmentations, followed by the generation of self-supervised signals. These signals are produced by maximizing the mutual information between the contrastive views. While this method has proven effective, we argue that current CL-based GCF models are still limited to current augmentation techniques. Existing data augmentation or noise perturbation may destroy the structural and semantic features of the original data and node attribute information is not considered. To tackle the above limitations, we propose a Meta-optimized Structure and Semantic Contrastive Learning for Graph Collaborative Filtering, named Meta-SSCL, which utilizes graph structure information and semantic information contrastive learning for recommendation. Specifically, we first model the structural and node semantic information representations with LightGCN and vanilla attention mechanism, respectively. Then consider the structural and semantic information as two contrastive views for recommendation. Next, the meta-optimized two-step training strategy generates adaptive contrastive views. Finally, we fuse structural and semantic representations for recommendation. Extensive experiments on real-world datasets demonstrate that Meta-SSCL consistently outperforms state-of-the-art sequential recommendation methods. The code is available1.

The rapid evolution of Large Language Models (LLMs) like OpenAI’s ChatGPT has significantly influenced the landscape of natural language processing, heralding breakthroughs in text generation capabilities. These advancements, while beneficial, have raised concerns regarding the authenticity and origin of LLM-generated content, particularly in the context of misuse such as misinformation dissemination and academic dishonesty. Addressing these concerns, this paper introduces a novel, customizable text watermarking technique that embeds a spectrum of information within LLM-generated texts. Unlike traditional binary classifiers that only differentiate between human and model-generated texts, our method embeds a watermark containing detailed information like the originating model, generation date, and user identifiers. This approach not only aids in distinguishing between human and LLM-generated texts but also carries crucial metadata for traceability and authenticity verification. Utilizing Quadrature Phase Shift Keying (QPSK) for embedding and decoding watermarks, our watermark method demonstrates effectiveness, secrecy, robustness, and customizability. Our text watermarking method maintains high-quality text while achieving a detection performance with an Area Under the Curve (AUC) score of 0.98, on par with advanced watermarking techniques on LLMs. Additionally, it exhibits a near-perfect success rate of almost 100% in decoding single-letter watermark information. Furthermore, it ensures robustness against common textual attacks, effectively balancing the need for traceability and authenticity verification with minimal impact on text quality. This method’s efficacy demonstrates its potential as a valuable tool in addressing the challenges posed by the misuse of generative AI technologies.

Conventional text embeddings, typically learned in the Euclidean space, may struggle to effectively capture word semantics based solely on the directional similarity between word vectors. In contrast, spherical text embeddings have demonstrated remarkable efficacy in various natural language processing (NLP) tasks recently. However, training spherical text generative models (STGMs) requires large representative datasets, which could potentially contain sensitive private information. To mitigate this concern, we propose a novel approach: the differential private spherical text generative model (DP-STGM), which facilitates learning text embeddings within the spherical space while ensuring privacy via efficient Riemannian optimization and the framework of differential privacy. To evaluate the efficacy of our privacy-preserving algorithm, we initially train an adversary using an external dataset without the application of differential privacy. Subsequently, we introduce two metrics to measure the model’s ability to protect privacy: (1) cosine similarity between recovered words from the adversary that generates the spherical text embeddings and those generated by DP-STGM, and (2) Top-n rank correlation. Our experimental findings demonstrate that DP-STGM outperforms baseline models, showcasing its superior performance. By leveraging the power of differential privacy in the Riemannian optimization process, our model achieves better preservation of sensitive information while simultaneously capturing the semantic nuances inherent in word embeddings. As a result, DP-STGM represents a robust and efficient solution for NLP tasks that require privacy protection without compromising on the quality of learned text embeddings. By offering a privacy-preserving alternative, DP-STGM broadens the range of applications for which STGMs can be safely employed, ensuring data privacy while harnessing the rich information contained in spherical text embeddings. Our work opens up new avenues for future research in privacy-aware NLP and advances the state-of-the-art in both privacy protection and semantic learning in this domain.

Sparse online learning has received extensive attention during the past few years. Most of existing algorithms that utilize ℓ1-norm regularization or ℓ1-ball projection assume that the feature space is fixed or changes by following explicit constraints. However, this assumption does not always hold in many real applications. Motivated by this observation, we propose a new online learning algorithm tailored for data streams described by open feature spaces, where new features can be occurred, and old features may be vanished over various time spans. Our algorithm named RSOL provides a strategy to adapt quickly to such feature dynamics by encouraging sparse model representation with an ℓ1- and ℓ2 -mixed regularizer. We leverage the proximal operator of the ℓ1,2 -mixed norm and show that our RSOL algorithm enjoys a closed-form solution at each iteration. A sub-linear regret bound of our proposed algorithm is guaranteed with a solid theoretical analysis. Empirical results benchmarked on nine streaming datasets validate the effectiveness of the proposed RSOL method over three state-of-the-art algorithms.

The user purchase behaviors are mainly influenced by their intentions (e.g., buying clothes for decoration, buying brushes for painting, etc.). Modeling a user's latent intention can significantly improve the performance of recommendations. Previous works model users' intentions by considering the predefined label in auxiliary information or introducing stochastic data augmentation to learn purposes in the latent space. However, the auxiliary information is sparse and not always available for recommender systems, and introducing stochastic data augmentation may introduce noise and thus change the intentions hidden in the sequence. Therefore, leveraging user intentions for sequential recommendation (SR) can be challenging because they are frequently varied and unobserved. In this paper, Intent contrastive learning with Cross Subsequences for sequential Recommendation (ICSRec) is proposed to model users' latent intentions. Specifically, ICSRec first segments a user's sequential behaviors into multiple subsequences by using a dynamic sliding operation and takes these subsequences into the encoder to generate the representations for the user's intentions. To tackle the problem of no explicit labels for purposes, ICSRec assumes different subsequences with the same target item may represent the same intention and proposes a coarse-grain intent contrastive learning to push these subsequences closer. Then, fine-grain intent contrastive learning is mentioned to capture the fine-grain intentions of subsequences in sequential behaviors. Extensive experiments conducted on four real-world datasets demonstrate the superior performance of the proposed ICSRec model compared with baseline methods.

Smart contracts hold billions of dollars in digital currency, and their security vulnerabilities have drawn a lot of attention in recent years. Traditional methods for detecting smart contract vulnerabilities rely primarily on symbol execution, which makes them time-consuming with high false positive rates. Recently, deep learning approaches have alleviated these issues but still face several major limitations, such as lack of interpretability and susceptibility to evasion techniques. In this paper, we propose a feature selection method for uplifting modeling. The fundamental concept of this method is a feature selection algorithm, utilizing interpretation outcomes to select critical features thereby reducing the scales of features. The learning speed could be accelerated significantly because of the reduction of the feature size. The experiment shows that our proposed model performs well in six types of vulnerability detection. The accuracy of each is higher than 93% and the average detection time of each smart contract is less than 1 ms. Notably, through our proposed feature selection algorithm, the training time of each type of vulnerability is reduced by nearly 80% compared with its original.

Existing methods for burned area segmentation (BAS) in optical remote sensing images (ORSIs) mainly adopt convolution neural network (CNN) as the backbone, which has limited receptive filed and suffers from long-term dependencies problem. To address this issue, we propose a novel salient object detection (SOD) network (DBINet) based on dual interactive convolution-transformer backbone for BAS in ORSIs. DBINet combines the benefits of CNN and transformer: CNN is good at extracting local spatial information while transformer does well in modeling long-term dependencies. The core component of DBINet is three newly designed modules: dual-feature fusion module (DFM), Convertor and a novel decoder. Specifically, DFM is proposed to bridge two different backbones. Convertor is designed to fuse the multi-scale coarse features from the main encoder and produce the fined feature for the decoder. The decoder has a multi-level feature aggregating process and a self-refining process, which restores the resolution and generates the prediction results. Experiments on three datasets demonstrate that our DBINet outperforms the state-of-the-art methods, and achieves the best S-measure on three datasets: 0.847, 0.888 and 0.883. Code is available at: https://github.com/Voruarn/DBINet.

Various correlations hidden in crowdsourcing annotation tasks bring opportunities to further improve the accuracy of label aggregation. However, these relationships are usually extremely difficult to be modeled. Most existing methods can merely make use of one or two correlations. In this paper, we propose a novel graph neural network model, namely LAGNN, which models five different correlations in crowdsourced annotation tasks by utilizing deep graph neural networks with convolution operations and derives a high label aggregation performance. Utilizing the group of high quality workers through labeling similarity, LAGNN can efficiently revise the preference among workers. Moreover, by injecting a little ground truth in its training stage, the label aggregation performance of LAGNN can be further significantly improved. We evaluate LAGNN on a large number of simulated datasets generated through varying six degrees of freedom and on eight real-world crowdsourcing datasets in both supervised and unsupervised (agnostic) modes. Experiments on data leakage is also contained. Experimental results consistently show that the proposed LAGNN significantly outperforms six state-of-the-art models in terms of label aggregation accuracy.

This article focuses on the task of Multi-Modal Summarization with Multi-Modal Output for China JD.COM e-commerce product description containing both source text and source images. In the context learning of multi-modal (text and image) input, there exists a semantic gap between text and image, especially in the cross-modal semantics of text and image. As a result, capturing shared cross-modal semantics earlier becomes crucial for multi-modal summarization. However, when generating the multi-modal summarization, based on the different contributions of input text and images, the relevance and irrelevance of multi-modal contexts to the target summary should be considered, so as to optimize the process of learning cross-modal context to guide the summary generation process and to emphasize the significant semantics within each modality. To address the aforementioned challenges, Multization has been proposed to enhance multi-modal semantic information by multi-contextually relevant and irrelevant attention alignment. Specifically, a Semantic Alignment Enhancement mechanism is employed to capture shared semantics between different modalities (text and image), so as to enhance the importance of crucial multi-modal information in the encoding stage. Additionally, the IR-Relevant Multi-Context Learning mechanism is utilized to observe the summary generation process from both relevant and irrelevant perspectives, so as to form a multi-modal context that incorporates both text and image semantic information. The experimental results in the China JD.COM e-commerce dataset demonstrate that the proposed Multization method effectively captures the shared semantics between the input source text and source images, and highlights essential semantics. It also successfully generates the multi-modal summary (including image and text) that comprehensively considers the semantics information of both text and image.

In biochemistry, graph structures have been widely used for modeling compounds, proteins, functional interactions, etc. A common task that divides these graphs into different categories, known as graph classification, highly relies on the quality of the representations of graphs. With the advance in graph neural networks, message-passing-based methods are adopted to iteratively aggregate neighborhood information for better graph representations. These methods, though powerful, still suffer from some shortcomings. The first challenge is that pooling-based methods in graph neural networks may sometimes ignore the part-whole hierarchies naturally existing in graph structures. These part-whole relationships are usually valuable for many molecular function prediction tasks. The second challenge is that most existing methods do not take the heterogeneity embedded in graph representations into consideration. Disentangling the heterogeneity will increase the performance and interpretability of models. This paper proposes a graph capsule network for graph classification tasks with disentangled feature representations learned automatically by well-designed algorithms. This method is capable of, on the one hand, decomposing heterogeneous representations to more fine-grained elements, whilst on the other hand, capturing part-whole relationships using capsules. Extensive experiments performed on several public-available biochemistry datasets demonstrated the effectiveness of the proposed method, compared with nine state-of-the-art graph learning methods.

Graph neural network (GNN) provides a powerful expressive way to embed graph-structured data, which has been widely applied and spans many fields. This article found an interesting but unreasonable phenomenon in some classical GNNs, namely label confusion. Theoretically, the dependence between prediction label and ground-truth should gradually increase and tend to converge with training epochs. On the contrary, label confusion shows that the dependence between the predicted label and ground-truth is strong first and then weak. This article explains it from the perspective of feature-noise dependence. Specifically, traditional GNN prediction usually assumes that the ground-truth consists of a mutually independent prediction label (dependent on node features) and noise (independent of node features). However, the propagation aggregation mechanism of GNN will integrate irrelevant information from neighboring nodes into the prediction label, resulting in the prediction label no longer being completely dependent on the node feature, or the noise no longer being completely independent of the node feature. Hence, this article proposes a Label-Dependent GNN to alleviate this problem, called LDGNN. LDGNN mainly consists of two limitations, namely feature-noise (the difference between predicted label and ground-truth) independence, and expectation-variance (EV) separation. Specifically, LDGNN introduces the Hilbert-Schmidt independence criterion (HSIC) as a regularization to minimize the dependence between input features and noise. Note that the main reason for adopting HSIC is that it can measure the nonlinear relationship between any two spatial variables. In this way, HSIC can guide GNN to retain more label-dependence information. Then, LDGNN designs an EV separation to centralize nodes within a class and disperse them between classes to further retain label-dependence information. Through these two strategies, the GNN’s expression ability can be enhanced. Next, we theoretically prove the essential reason why LDGNN alleviates label confusion and has been verified in experiments. To verify the performance of LDGNN, we apply it to four classical GNN models on three datasets, and experimental results demonstrate the effectiveness of LDGNN.

Self-supervised contrastive learning has well advanced the development of bundle recommendation. However, self-supervised contrastive learning may misclassify some positive samples as negative samples, resulting in suboptimal models. Therefore, the industry has proposed supervised contrastive learning to alleviate this drawback. Inspired by this idea, we seek a more elegant contrastive learning paradigm in the field of recommendation, so we propose a dual-supervised contrastive learning for bundle recommendation (DSCBR), which integrates supervised and self-supervised contrastive learning to exploit the full potential of contrastive learning. Specifically, we first construct self-supervised contrastive learning between the two different views (bundle view and item view), which encourages the alignment of the two separately learned views and boosts the effectiveness of learned representations. Second, we use the interaction information to construct supervised contrastive learning and leverage the bundle–bundle cooccurrence graph for further enhancement. By introducing supervised contrastive learning, our model explicitly models user and bundle proximity in different views, improving the model’s robustness and generalization. Finally, we jointly perform self-supervised and supervised contrastive learning across multiple views. Extensive experiments on three public datasets demonstrate the effectiveness of our model.

Radar echo extrapolation, as a widely used approach for precipitation nowcasting, plays a pivotal role in severe convective weather warnings. Previous studies have encountered the dilemma of extrapolation ambiguity, which leads to low availability of extrapolation results. To get over the hurdle of extrapolation ambiguity, we propose a vivid radar network for precipitation nowcasting called VRNet. We first implement a multiscale spatial feature fusion module to extract richer spatial feature information, which contributes to producing clear images when extrapolating result reconstructions. Furthermore, we construct a generative adversarial network (GAN) with a ConvLSTM unit to enhance the model’s spatiotemporal information representation capability. In addition, a weighted loss function based on radar echo intensity is designed to address the distribution characteristics of radar echo intensity, improving the global averaging strategy of the mean squared error loss function. Experimental results demonstrate that the proposed model outperforms the benchmark models in the radar echo extrapolation task, which obtains a higher accuracy rate and improves the clarity of the extrapolated image.

Website Fingerprinting (WF) is a network traffic mining technique for anonymous traffic identification, which enables a local adversary to identify the target website that an anonymous network user is browsing. WF attacks based on deep convolutional neural networks (CNN) get the state-of-the-art anonymous traffic classification performance. However, due to the locality restriction of CNN architecture for feature extraction on sequence data, these methods ignore the temporal feature extraction in the anonymous traffic analysis. In this paper, we present Website Fingerprinting Transformer (WF-Transformer), a novel anonymous network traffic analysis method that leverages Transformer networks for temporal feature extraction of traffic traces and improves the classification performance of Tor encrypted traffic. The architecture of WF-Transformer is specially designed for traffic trace processing and can classify anonymous traffic effectively. Furthermore, we evaluate the performance of WF-Transformer in both closed-world and open-world scenarios. In the closed-world scenario, WF-Transformer attains 99.1% accuracy on Tor traffic without defenses, better than state-or-the-art attacks, and archives 92.1% accuracy on the traces defended by WTF-PAD method. In the open-world scenario, WF-Transformer has better precision and recall on both defended and non-defended traces. Furthermore, WF-Transformer with a short input length (2000 cells) outperforms the DF method with a long input length (5000 cells).

Graph collaborative filtering (GCF) has achieved exciting recommendation performance with its ability to aggregate high-order graph structure information. Recently, contrastive learning (CL) has been incorporated into GCF to alleviate data sparsity and noise issues. However, most of the existing methods employ random or manual augmentation to produce contrastive views that may destroy the original topology and amplify the noisy effects. We argue that such augmentation is insufficient to produce the optimal contrastive view, leading to suboptimal recommendation results. In this article, we proposed a L earnable M odel A ugmentation C ontrastive L earning (LMACL) framework for recommendation, which effectively combines graph-level and node-level collaborative relations to enhance the expressiveness of collaborative filtering (CF) paradigm. Specifically, we first use the graph convolution network (GCN) as a backbone encoder to incorporate multi-hop neighbors into graph-level original node representations by leveraging the high-order connectivity in user-item interaction graphs. At the same time, we treat the multi-head graph attention network (GAT) as an augmentation view generator to adaptively generate high-quality node-level augmented views. Finally, joint learning endows the end-to-end training fashion. In this case, the mutual supervision and collaborative cooperation of GCN and GAT achieves learnable model augmentation. Extensive experiments on several benchmark datasets demonstrate that LMACL provides a significant improvement over the strongest baseline in terms of Recall and NDCG by 2.5%–3.8% and 1.6%–4.0%, respectively. Our model implementation code is available at https://github.com/LiuHsinx/LMACL .

Sequential recommendation aims to predict the next item of interest to users based on their historical behavior data. Usually, users’ global and local preferences jointly affect the final recommendation result in different ways. Most existing works use transformers to globally model sequences, which makes them face the dilemma of quadratic computational complexity when dealing with long sequences. Moreover, the scope setting of the user’s local preference is usually static and single, and cannot cover richer multi-level local semantics. To this end, we proposed a parallel architecture for capturing global representation and M ulti-granularity L ocal dependencies with M LP for sequential Rec ommendation ( MLM4Rec ). For global representation, we utilize modified MLP-Mixer to capture global information of user sequences due to its simplicity and efficiency. For local representation, we incorporate convolution into MLP and propose a multi-granularity local awareness mechanism for capturing richer local semantic information. Moreover, we introduced a weight pooling method to adaptively fuse local-global representations instead of directly concatenation. Our model has the advantages of low complexity and high efficiency thanks to its simple MLP structure. Experimental results on three public datasets demonstrate the effectiveness of our proposed model. Our code is available here 1 .

Local spatial-aware community detection refers to detecting a spatial-aware community for a given node using local information. A spatial-aware community means that nodes in the community are tightly connected in structure, and their locations are close to each other. Existing studies focus on detecting the local non-overlapping spatial-aware community, i.e., detecting a spatial-aware community containing the given node. However, many geosocial networks often contain overlapping spatial-aware communities. Therefore, we propose a local overlapping spatial-aware community detection (LOSCD) problem, which aims to detect all spatial-aware communities that contain a given node with local information. To address LOSCD problem, we design an algorithm based on Spatial Modularity and Edge Similarity, called SMES. SMES contains two processes: spatial expansion and structure detection. The spatial expansion process involves using spatial modularity to identify nodes that are spatially close, while the structural detection process employs edge similarity to identify nodes that are structurally close. Experimental results demonstrate that SMES outperforms comparison algorithms in terms of both structural and spatial cohesiveness.

Object detection is a widely studied problem in existing works. However, in this paper, we turn to a more challenging problem of “ Covered Object Reasoning ”, aimed at reasoning the category label of target object in the given image particularly when it has been totally covered (or invisible ). To resolve this problem, we propose CoBjeason to seize the opportunity when visual reasoning meets the knowledge graph, where “ empirical cognition ” on common visual contexts have been incorporated as knowledge graph to conduct reinforced multi-hop reasoning via two collaborative agents. Such two agents, for one thing, stand at the covered object (or unknown entity ) to observe the surrounding visual cues in the given image and gradually select entities and relations from the global gallery-level knowledge graph which contains entity-pairs frequently occurring across the entire image-collection, so as to infer the main structure of image-level knowledge graph forward expanded from the unknown entity . In turn, for another, based on the reasoned image-level knowledge graph, the semantic context among entities will be aggregated backward into unknown entity to select an appropriate entity from the global gallery-level knowledge graph as the reasoning result. Moreover, such two agents will collaborate with each other, securing that the above Forward & Backward Reasoning will step towards the same destination of the higher performance on covered object reasoning. To our best knowledge, this is the first work on Covered Object Reasoning with Knowledge Graphs and reinforced Multi-Agent collaboration. Particularly, our study on Covered Object Reasoning and the proposed model CoBjeason could offer novel insights into more basic Computer Vision (CV) tasks, such as Semantic Segmentation with better understanding on the current scene when some objects are blurred or covered, Visual Question Answering with enhancement on the inference in more complicated visual context when some objects are covered or invisible, and Image Caption Generation with the augmentation on the richness of visual context for images containing partially visible objects. The improvement on the above basic CV tasks can further refine more complicated ones involved with nuanced visual interpretation like Autonomous Driving, where the recognition and reasoning on partially visible or covered object are critical. According to the experimental results, our proposed CoBjeason can achieve the best overall ranking performance on covered object reasoning compared with other models, meanwhile enjoying the advantage of lower “ exploration cost ”, with the insensitivity against the long-tail covered objects and the acceptable time complexity.

Online shopping has become a crucial way to encourage daily consumption, where the User-generated, or crowdsourced product comments, can offer a broad range of feedback on e-commerce products. As a result, integrating critical opinions or major attitudes from the crowdsourced comments can provide valuable feedback for marketing strategy adjustment or product-quality monitoring. Unfortunately, the scarcity of annotated ground truth on the integrated comment, or the limited gold integration reference, has incurred the infeasibility of the regular supervised-learning-based comment integration. To resolve this problem, in this article, inspired by the principle of Transfer Learning, we propose a three-stage transferable and generative crowdsourced comment integration framework ( TTGCIF ) based on zero-and-few-shot learning with the support of domain distribution alignment. The proposed framework aims at generating abstractive integrated comment in target domain via the enhanced neural text generation model, by referring the available integration resource in related source domains, to avoid the exhausted effort on resource annotation devoted to the target domain. Specifically, at the first stage, to enhance the domain transferability, representations on the crowdsourced comments have been aligned up between the source and target domain, by minimizing the domain distribution discrepancy in the kernel space. At the second stage, Zero-shot comment integration mechanism has been adopted to deal with the dilemma that none of the gold integration reference may be available in target domain. In other words, taking the sample-level semantic prototype as input, the enhanced neural text generation model in TTGCIF is trained to learn data semantic association among different domains via semantic prototype transduction, so that the “ unlabeled ” crowdsourced comments in target domain can be associated with existing integration references in related source domains. At the third stage, based on the parameters trained at the second stage, fast domain adaptation mechanism in a Few-shot manner has also been adopted by seeking most potential parameters along the gradient direction constrained by instances across multiple source domains. In this way, parameters in TTGCIF can be sensitive to any alteration on training data, ensuring that even if only few annotated resource in target domain are available for “Fine-tune,” TTGCIF can still react promptly to achieve effective target domain adaptation. According to the experimental results, TTGCIF can achieve the best transferable product comment integration performance in target domain, with fast and stable domain adaption effect depending on no more than 10% annotated resource in target domain. More importantly, even if TTGCIF has not been fine-tuned on the target domain, yet by referring to the available integration resource in related source domains, the integrated comments generated by TTGCIF on the target domain are still superior to those generated by models already fine-tuned on the target domain.

Contrastive learning with Transformer-based sequence encoder has gained predominance for sequential recommendation due to its ability to mitigate the data noise and the data sparsity issue. However, existing contrastive learning approaches for sequential recommendation still suffer from two limitations. First, they mainly center on left-to-right unidirectional Transformers as base encoders, which are suboptimal for sequential recommendation because user behaviors may not be a rigid left-to-right sequence. Second, they devise contrastive learning objectives only from the sequence level, neglecting the rich self-supervision signals from the feature level. To address these limitations, we propose a novel framework called Feature-aware Contrastive Learning with bidirectional Transformers for sequential Recommendation (FCLRec) to effectively leverage feature information for sequential recommendation. Specifically, we first augment bidirectional Transformers with a novel feature-aware self-attention module that is able to simultaneously model the complex relationships between sequences and features. Next, we propose a novel feature-aware contrastive learning objective that generates a collection of positive samples via three types of augmentations from three different levels. Finally, we adopt feature prediction as an auxiliary task to strengthen the connections between items and features. Our experimental results on four public benchmark datasets show that FCLRec outperforms the state-of-the-art methods for sequential recommendation.

Sequential Recommendation (SR) methods play a crucial role in recommender systems, which aims to capture users' dynamic interest from their historical interactions. Recently, Contrastive Learning (CL), which has emerged as a successful method for sequential recommendation, utilizes various data augmentations to generate contrastive views to mine supervised signals from data to alleviate data sparsity issues. However, most existing sequential data augmentation methods may destroy semantic sequential interaction characteristics. Meanwhile, they often adopt random operations when generating contrastive views leading to suboptimal performance. To this end, in this paper, we propose a Learnable Model Augmentation Contrastive learning for sequential Recommendation (LMA4Rec). Specifically, LMA4Rec first takes the model-based augmentation method to generate constructive views. Then, LMA4Rec uses Learnable Bernoulli Dropout (LBD) to implement learnable model augmentation operations. Next, contrastive learning is used between the contrastive views to extract supervised signals. Furthermore, a novel multi-positive contrastive learning loss alleviates the supervised sparsity issue. Finally, experiments on public datasets show that our LMA4Rec method effectively improved sequential recommendation performance compared with the state-of-the-art baseline methods.

Owing to the lack of a universal definition of communities, some semi-supervised community detection approaches learn the concept of community structures from known communities, and then dig out communities using learned concepts of communities. In some cases, users are only interested in the community containing a given node. However, communities detected by these semi-supervised approaches may not contain a given node. Besides, these methods traverse the entire network to detect many communities and cost more resources than a local algorithm. Therefore, it is necessary and meaningful to find the local community that contains a given node with prior information on the local network around the given node. We call this a Semi-supervised Local Community Detection (SLCD) problem. In this paper, prior information refers to certain known communities. To address the SLCD problem, we propose the Semi-supervised Local community detection with the Structural Similarity algorithm, called SLSS, which uses some known communities instead of all known communities. The idea of SLSS is to use the structural similarity between the known communities and the detected community, calculated by the graph kernel, to guide the expansion of the community. Experimental results show that SLSS outperforms other algorithms on six real-world datasets.

The identification of the local community structure in geo-social networks has been gaining increasing attention. The structure of geo-social networks evolves over time with the addition/deletion of edges/nodes and the update of node locations, which has motivated recent studies to mine local communities in dynamic geo-social networks. Mining communities in evolving geo-social networks is essential for understanding the evolution of group behaviors. However, in most previous studies on the community mining in dynamic networks, local spatial-aware communities were not identified in evolving geo-social networks. Therefore, in this study, the problem of determining local spatial-aware communities in evolving geo-social networks is proposed. To address this problem, we propose a parameter-free algorithm, called LSADEN. Specifically, LSADEN involves two main steps: i) selecting candidate nodes, where LSADEN defines the community dominance relation under dynamic environments to obtain candidate nodes that improve the community in terms of the community quality or the smoothness between communities at adjacent time stamps; ii) community expansion, where LSADEN designs the Manhattan distance of communities to add some candidate nodes to the local community. Experimental results on six real-world datasets and one synthetic dataset show that LSADEN performs well both in terms of the quality of communities and the smoothness between communities at adjacent time stamps.

Nowadays, crowdsourced documents like Wikipedia pages and comments on products are all over the Internet. However, documents generated by crowdsourcing participants may contain inconsistent facts, implicit semantics and fabricated contents, thus threatening the trustworthiness of information content security available in Internet. To address this problem, we propose FuFaction, enabled by an enhanced observation mechanism based on the notion of hybrid-mask consisting of a hard-mask and a soft-mask, to eliminate factual inconsistencies on crowdsourced documents at the hidden-state level (or in a fuzzy way), according to the given evidence retrieved from an external open domain. Specifically, instead of focusing on a specific category of factual inconsistency, FuFaction captures anomalous hidden-states between a crowdsourced document and evidence obtained via a reverse-attention mechanism, where a hard-mask controls the attending direction as bidirectional and unidirectional for better understanding on semantics. Then, a soft-mask is generated with the help of the hard-masked reverse-attention to revise or mask anomalous hidden-states on the crowdsourced document. Afterwards, the masked hidden-states are further refined by a cross reverse-attention and factual consistency reinforcement strategy, based on which a new crowdsourced document with higher factual consistency is generated via neural text generation. According to our experimental results, FuFaction can effectively deal with the fuzzy factual inconsistencies on crowdsourced documents, achieving the overall best performance in terms of factual consistency metrics with a little higher (yet still competitive) editing cost on literal vocabulary, so as to reflect factually consistent semantics supported by the given evidence.

Deep learning has been widely adopted in Android malicious software (malware) detection. However, poor explanation in deep learning-based detection models severely undermines user trusts and poses a significant obstacle to their practical promotion in critical security domains. Some studies strive to uncover the rationale behind a model's decision. Unfortunately, these efforts are often hindered by the limitations of feature extraction methods, such as primarily relying on static analysis to derive separate and approximate behavioral descriptions of applications (apps). As a result, establishing a reliable interpretation for deep learning-based malware detection models remains an open issue. In this work, we propose a novel framework XDeepMal to interpret deep learning-based malware detection models. Specifically, in XDeepMal, we formulate a dynamic analysis tool XTracer+to capture runtime behaviors of apps and automatically generate their continuous behavior trajectories. Then, we propose a novel interpreter to pinpoint certainty behavior fragments that are crucial for deep learning models to make their decisions. This approach regards the identification of the most critical fragments as an optimization problem and leverages heuristic algorithms for implementation. We conduct extensive experiments on a real-world dataset to investigate the effectiveness and reliability of XDeepMal. These experiments cover intuitive case studies (malware family and individual app) and in-depth quantitative analysis. Additionally, we evaluate its coverage and efficiency. Our experimental results demonstrate that XDeepMal is capable of generating convincing interpretations for deep learning (e.g., Transformer) based models within feasible inference time, which greatly benefits security analysts in accurately comprehending why an app is identified as malware by deep learning-based detection models.

Multilabel annotation is a critical step to generate training sets when learning classification models in various application domains, but asking domain experts to provide labels is usually time-consuming and expensive, which cannot meet the current requirement of the fast evolution of the models in the big data era. Although crowdsourcing provides a fast solution to acquire labels for multilabel learning, it faces the risk of high data acquisition cost and low label quality. This article proposes a novel one-coin label-dependent active crowdsourcing (OCLDAC) method to iteratively query noisy labels from crowd workers and learn multilabel classification models. In each iteration of active learning, integrated labels of instances are first inferred by a novel one-coin label-dependent model, which utilizes a mixture of multiple independent Bernoulli distributions to explore and exploit correlations among the labels to increase the accuracy of truth inference. Then, instances, labels, and workers are selected according to the novel strategies that incorporate the distribution of noisy labels, the prediction probability of learning models, label correlations, and the reliability of crowd workers. Simulations on eight multilabel datasets and evaluation on one real-world crowdsourcing dataset consistently show that the proposed OCLDAC significantly outperforms the state-of-the-art methods and their variants.

Federated learning (FL) is vulnerable to data poisoning attacks when an adversary attempts to upload poison gradients with the intent to corrupt the global model of FL. Various approaches have been proposed to counter these risks. However, it becomes challenging when one tries to preserve the privacy of FL participants and ensure robustness against data poisoning attacks. In this paper, we propose DPFLA, a novel scheme that can detect poisoning attacks without revealing the actual gradients of participants. DPFLA is a lossless aggregation scheme delicately designed for adopting masks to protect private data while extracting poisoned data features. Specifically, we first apply removable masks to the gradients outputted by each participant. Second, we aggregate the masked data and decompose them using Singular Value Decomposition (SVD) to extract specific features as well as achieve dimensionality reduction. Third, we leverage a clustering paradigm to detect poison gradients from the low dimension and eliminate them in the following training rounds. We conducted extensive experiments to demonstrate that DPFLA can detect poison gradients effectively. Additionally, the comparisons of case studies demonstrate that DPFLA outperforms the state-of-the-art methods.

While the Android platform is gaining explosive popularity, the number of malicious software (malware) is also increasing sharply. Thus, numerous malware detection schemes based on deep learning have been proposed. However, they are usually suffering from the cumbersome models with complex architectures and tremendous parameters. They usually require heavy computation power support, which seriously limit their deployment on actual application environments with limited resources (e.g., mobile edge devices). To surmount this challenge, we propose a novel Knowledge Distillation (KD) structure—Knowledge Search (KS). KS exploits Neural Architecture Search (NAS) to adaptively bridge the capability gap between teacher and student networks in KD by introducing a parallelized student-wise search approach. In addition, we carefully analyze the characteristics of malware and locate three cost-effective types of features closely related to malicious attacks, namely, Application Programming Interfaces (APIs), permissions and vulnerable components, to characterize Android Applications (Apps). Therefore, based on typical samples collected in recent years, we refine features while exploiting the natural relationship between them, and construct corresponding datasets. Massive experiments are conducted to investigate the effectiveness and sustainability of KS on these datasets. Our experimental results show that the proposed method yields an accuracy of 97.89% to detect Android malware, which performs better than state-of-the-art solutions.

Smart contracts have gained extensive adoption across diverse industries, including finance, supply chain, and the Internet of Things. Nevertheless, the surge in security incidents of smart contracts over recent years has led to substantial economic losses. Therefore, ensuring the security of smart contracts has become a critical and complex challenge in both academic and industrial domains. Based on 539 real-world security incidents in the Ethereum platform and audit reports from 10 authoritative auditing institutions, we summarize 27 types of exploited security vulnerabilities and draw insights into their principles, typical cases, relevant research and recommended prevention strategies. Besides, we also gather 7 other potentially threatening vulnerability types as supplements. On this basis, we conduct an in-depth analysis of the root causes of vulnerabilities and further formulate eight safety practical rules. Moreover, we perform a comprehensive review of 178 recent papers on smart contract security analysis, classifying detection methods into formal verification, fuzz testing, machine learning, program analysis, and others. For each category, we seize the specific detection tools and analyze them comprehensively. Then, we conduct an extensive analysis and synthesis from various angles, presenting a comprehensive overview of the current research landscape in smart contract security detection. We also discuss current on-chain and off-chain repair methods. Finally, this review outlines major challenges and highlights potential areas for future research in this field.

Deep learning models have shown great performances in natural language processing tasks. While much attention has been paid to improvements in utility, privacy leakage and social bias are two major concerns arising in trained models. In order to tackle these problems, we protect individuals' sensitive information and mitigate gender bias simultaneously. First, we propose a selective privacy-preserving method that only obscures individuals' sensitive information. Then we propose a negative multi-task learning framework to mitigate the gender bias which contains a main task and a gender prediction task. We analyze two existing word embeddings and evaluate them on sentiment analysis and a medical text classification task. Our experimental results show that our negative multi-task learning framework can mitigate the gender bias while keeping models’ utility.

Crowdsourcing is a popular method for crowd workers to collaborate on tasks. However, workers coordinate and share answers during the crowdsourcing process. The term for this is "collusion". Copies from others and repeated submissions are detrimental to the quality of the assignments. The majority of the existing research on collusion detection is limited to ground truth problems (e.g., labeling tasks) and requires a predetermined threshold to be established in advance. In this paper, we aim to detect collusion behavior of workers in an adaptive way, and propose an Adaptive Clustering Based Collusion Detection approach (ACCD) for a broad range of task types and data types solved via crowdsourcing (e.g., continuous rating with or without distributions). Extensive experiments on both real-world and synthetic datasets show the superiority of ACCD over state-of-the-art approaches.

We aim to propose a system repairing programs with logic errors to be functionally correct among different programming languages. Logic error program repair has always been a thorny problem: First, a logic error is usually harder to repair than a syntax error in a program because it has no diagnostic feedback from compilers. Second, it requires inferring in different ranges (i.e., the distance of related code lines) and tracking symbols across its pseudocode, source code, and test cases. Third, the logic error datasets are scarce, since an ideal logic error dataset should contain lots of components during the development procedure of a program, including a program specification, pseudocode, source code, test cases, and test reports (i.e., test case failure report). In our work, we propose novel solutions to these challenges. First, we introduce pseudocode information to assist logic error localization and correction. We construct a code-pseudocode graph to connect symbols across a source code and its pseudocode and then apply a graph neural network to localize and correct logic errors. Second, we collect logic errors generated in the process of syntax error repairing via DrRepair from 500 programs in the SPoC dataset and reconstruct them to our single logic error dataset, which we leverage to train and evaluate our models. Our experimental results show that we achieve 99.39% localization accuracy and 19.20% full repair accuracy on logic errors with five-fold cross-validation. Based on our current work, we will replenish and construct more complete public logic error datasets and propose a novel system to comprehend different programming languages from several perspectives and correct logic errors to be functionally correct.

In this paper, we measure the privacy leakage via studying whether graph representations can be inverted to recover the graph used to generate them via graph reconstruction attack (GRA). We propose a GRA that recovers a graph's adjacency matrix from the representations via a graph decoder that minimizes the reconstruction loss between the partial graph and the reconstructed graph. We study three types of representations that are trained on the graph, i.e., representations output from graph convolutional network (GCN), graph attention network (GAT), and our proposed simplicial neural network (SNN) via a higher-order combinatorial Laplacian. Unlike the first two types of representations that only encode pairwise relationships, the third type of representation, i.e., SNN outputs, encodes higher-order interactions (e.g., homological features) between nodes. We find that the SNN outputs reveal the lowest privacy-preserving ability to defend the GRA, followed by those of GATs and GCNs, which indicates the importance of building more private representations with higher-order node information that could defend the potential threats, such as GRAs.

Although recent network representation learning (NRL) works in text-attributed networks demonstrated superior performance for various graph inference tasks, learning network representations could always raise privacy concerns when nodes represent people or human-related variables. Moreover, standard NRLs that leverage structural information from a graph proceed by first encoding pairwise relationships into learned representations and then analysing its properties. This approach is fundamentally misaligned with problems where the relationships involve multiple points, and topological structure must be encoded beyond pairwise interactions. Fortunately, the machinery of topological data analysis (TDA) and, in particular, simplicial neural networks (SNNs) offer a mathematically rigorous framework to evaluate not only higher-order interactions, but also global invariant features of the observed graph to systematically learn topological structures. It is critical to investigate if the representation outputs from SNNs are more vulnerable compared to regular representation outputs from graph neural networks (GNNs) via pairwise interactions. In my dissertation, I will first study learning the representations with text attributes for simplicial complexes (RT4SC) via SNNs. Then, I will conduct research on two potential attacks on the representation outputs from SNNs: (1) membership inference attack, which infers whether a certain node of a graph is inside the training data of the GNN model; and (2) graph reconstruction attacks, which infer the confidential edges of a text-attributed network. Finally, I will study a privacy-preserving deterministic differentially private alternating direction method of multiplier to learn secure representation outputs from SNNs that capture multi-scale relationships and facilitate the passage from local structure to global invariant features on text-attributed networks.

This research introduces a system that combines Graph Neural Networks and Machine Learning to accurately pinpoint logic errors in programming. Unlike syntax errors which compilers can easily identify, logic errors lack clear diagnostic feedback, making them harder to locate. To enhance error detection, our system incorporates pseudocode, enabling a structured connection between source code and its pseudocode through a code-pseudocode graph. By using a Graph Neural Network model, we can localize these logic errors with higher precision. To train and validate our model, we gather logically erroneous programs from the syntax error repair process, forming a unique dataset. Experiments reveal an impressive localization accuracy of 99.42% on this dataset with five-fold cross-validation. Future directions include expanding our dataset and devising a system to interpret diverse programming languages for efficient error localization.

Language models have exhibited remarkable performance across diverse tasks, including those in the realm of biological research such as protein language modeling. Transcription factors (TFs) are pivotal in gene regulation, influencing gene expression through specific DNA sequence binding. While various TF prediction techniques exist, they often necessitate extensive training datasets or suffer from limited accuracy. In this study, we propose an ESM-TFpredict model, which leverages a pre-trained protein language model to encode amino acid sequences, followed by 1-D convolutional neural networks for TF prediction. To elucidate the model’s decision-making, we employ an integrated gradients method to highlight the important features driving TF identification. Comparative experimental analysis with existing models, DeepTFactor and TFpredict, reveals that the ESM-TFpredict achieves an accuracy exceeding 95% across four evaluation metrics, surpassing both competitors. By utilizing a slide window approach for protein representation compression, the training duration of ESM-TFpredict is 315.78 seconds, which is only 51% of the training time required by DeepTFactor and a mere 12% of the training time required by TFpredict. We further analyze the contributions of known TF-related regions (average attribution score 0.9152) versus Non-TF-related regions (average attribution score 0.0848), demonstrating that the TF-related regions have dominant influences on TF prediction.

Online learning updates models incrementally with new data, avoiding large storage requirements and costly model recalculations. In this paper, we introduce “OLR-WA; OnLine Regression with Weighted Average”, a novel and versatile multivariate online linear regression model. We also investigate scenarios involving drift, where the underlying patterns in the data evolve over time, conduct convergence analysis, and compare our approach with existing online regression models. The results of OLR-WA demonstrate its ability to achieve performance comparable to the batch regression, while also showcasing comparable or superior performance when compared with other state-of-the-art online models, thus establishing its effectiveness. Moreover, OLR-WA exhibits exceptional performance in terms of rapid convergence, surpassing other online models with consistently achieving high r2values as a performance measure from the first iteration to the last iteration, even when initialized with minimal amount of data points, as little as 1% to 10% of the total data points. In addition to its ability to handle time-based (temporal drift) scenarios, remarkably, OLR-WA stands out as the only model capable of effectively managing confidence-based challenging scenarios. It achieves this by adopting a conservative approach in its updates, giving priority to older data points with higher confidence levels. In summary, OLR-WA’s performance further solidifies its versatility and utility across different contexts, making it a valuable solution for online linear regression tasks.

The rapid advancement of neural networks has permeated industries worldwide. However, this remarkable success often faces adversity while deploying on devices with limited resources due to the high computational power demands and storage requirements. To address these challenges, a model compressing technique such as knowledge distillation (KD) has emerged as an effective process for training compact models. However, due to the lack of interpretability, knowledge distillation is often hard to explain. In this paper, our focus is to implement a knowledge distillation framework that can provide interpretability alongside improved performance. With the help of our experiments, we demonstrate that this can be achieved by integrating first-order logical formulas into a neurosymbolic learning approach within the knowledge distillation framework. Diverging from the prior research, in this paper, we introduce a Neurosymbolic Knowledge Distillation Framework (KD-LTN), which is a composition of Logic Tensor Network (LTN) and Knowledge Distillation (KD). Notably, our KD-LTN network not only enhances interpretability but also achieves accuracy improvements, compared with the conventional knowledge distillation framework.

Simplicial neural networks (SNNs) were proposed to generate higher-order simplicial complex representations as vectors that encode not only pairwise relationships but also higher-order interactions between nodes. Although these vectors allowing us to consider richer data representations compared to typical graph convolution, most real-world graphs associated with molecule or human-related activities are often sensitive and might contain confidential information, e.g., molecular geometry or friend lists. However, little works investigate the potential threats for these simplicial complexes (higher-order interactions between nodes). We name this threat by Simplicial Complexes Reconstruction Attack (SCRA) and conduct this attack by studying whether the vectors can be inverted to (approximately) recover the simplicial complexes who used to generate them. Specifically, we first generate the vectors via a k-simplex2vec approach that extends the node2vec algorithm to simplices of higher dimensions to associate Euclidean vectors to simplicial complexes. We then present a Simplex2vec Backward algorithm to perform the SCRA on k-simplex2vec vectors by pointwise mutual information (PMI) matrix reconstruction.

Releasing the representations of nodes in real-world graphs associated with people or human-related activities, such as social and economic networks, gives adversaries a potential way to infer the sensitive information of edges. For example, graph convolutional layers initially aggregate node representations with their neighbors before passing them through non-linear activation functions. Hence, the released node representations may potentially breach edge privacy of the node neighbors. Thus, in this work, we study whether representations can be inverted to recover the graph used to generate them. We study three types of outputs that are trained on the graph, i.e., representations output from graph convolutional networks (GCNs), representations output from graph attention networks (GATs), and representations output from our proposed simplicial neural networks (SNNs). Unlike the first two types of representations that only encode pairwise relationships, the third type of representation, i.e., SNN outputs, encodes higher-order interactions (e.g., homological features) between nodes. We propose two graph reconstruction attacks (GRAs), i.e., Type-1 and Type-2 attacks, to recover a graph’s adjacency matrix from the three types of outputs trained on the graph. Specifically, our GRAs utilize a graph-decoder to minimize the reconstruction loss for the generated adjacency matrix via back-propagation. Our conclusions are two folds. First, our Type-2 attack achieves the best performance among all current GRAs. Second, we find that GCN outputs obtain the least precision and AUC on five datasets, followed by the GAT outputs, followed by the SNN outputs. Therefore, the SNN outputs reveal the lowest privacy-preserving ability to defend the GRAs. We further propose an unbiased multi-bit rectifier, by which the server can communicate with the nodes to privately collect their representations to defend the GRAs from potential adversaries.

Federated learning (FL) is a promising paradigm to enable collaborative model training with decentralized data.However, the training process of Large Language Models (LLMs) generally incurs the update of significant parameters, which limits the applicability of FL techniques to tackle the LLMs in real scenarios.Prompt tuning can significantly reduce the number of parameters to update, but it either incurs performance degradation or low training efficiency.The straightforward utilization of prompt tuning in the FL often raises non-trivial communication costs and dramatically degrades performance.In addition, the decentralized data is generally non-Independent and Identically Distributed (non-IID), which brings client drift problems and thus poor performance.This paper proposes a Parameter-efficient prompt Tuning approach with Adaptive Optimization, i.e., Fed-PepTAO, to enable efficient and effective FL of LLMs.First, an efficient partial prompt tuning approach is proposed to improve performance and efficiency simultaneously.Second, a novel adaptive optimization method is developed to address the client drift problems on both the device and server sides to enhance performance further.Extensive experiments based on 10 datasets demonstrate the superb performance (up to 60.8% in terms of accuracy) and efficiency (up to 97.59% in terms of training time) of FedPepTAO compared with 9 baseline approaches.Our code is available at https://github.com/llm-eff/FedPepTAO.

Sequential recommendation (SR) aims to model user preferences by capturing behavior patterns from their item historical interaction data. Most existing methods model user preference in the time domain, omitting the fact that users’ behaviors are also influenced by various frequency patterns that are difficult to separate in the entangled chronological items. However, few attempts have been made to train SR in the frequency domain, and it is still unclear how to use the frequency components to learn an appropriate representation for the user. To solve this problem, we shift the viewpoint to the frequency domain and propose a novel Contrastive Enhanced SLIde Filter MixEr for Sequential Recommendation, named SLIME4Rec. Specifically, we design a frequency ramp structure to allow the learnable filter slide on the frequency spectrums across different layers to capture different frequency patterns. Moreover, a Dynamic Frequency Selection (DFS) and a Static Frequency Split (SFS) module are proposed to replace the self-attention module for effectively extracting frequency information in two ways. DFS is used to select helpful frequency components dynamically, and SFS is combined with the dynamic frequency selection module to provide a more fine-grained frequency division. Finally, contrastive learning is utilized to improve the quality of user embedding learned from the frequency domain. Extensive experiments conducted on five widely used benchmark datasets demonstrate our proposed model performs significantly better than the state-of-the-art approaches. Our code is available at https://github.com/sudaada/SLIME4Rec.

Crowdsourcing is a sourcing model where individuals or organizations obtain goods and services from a large, relatively open and often rapidly evolving group of internet users. The most common way that crowdsourcing can facilitate machine learning is to annotate instances with labels [1] . However, the same instance may have inconsistent class labels, in the eyes of various annotators. Therefore, current efforts in crowdsourcing mainly focus on the truth inference or label integration, to remove inconsistent labels or to alleviate biased labeling. In turn, instances with the integrated labels could facilitate the training on machine learning models. The future direction of crowdsourcing is to apply more fine-grained truth inference methods to different application domains [2] . Consequently, we evolve toward another challenging problem of comment integration. That is, how can we integrate or summarize the core opinions of multiple product comments obtained from users, rather than the discrete labels.

Deep learning and symbolic learning are two frequently employed methods in Sequential Recommendation (SR). Recent neural-symbolic SR models demonstrate their potential to enable SR to be equipped with concurrent perception and cognition capacities. However, neural-symbolic SR remains a challenging problem due to open issues like representing users and items in logical reasoning. In this paper, we combine the Deep Neural Network (DNN) SR models with logical reasoning and propose a general framework named Sequential Recommendation with Probabilistic Logical Reasoning (short for SR-PLR). This framework allows SR-PLR to benefit from both similarity matching and logical reasoning by disentangling feature embedding and logic embedding in the DNN and probabilistic logic network. To better capture the uncertainty and evolution of user tastes, SR-PLR embeds users and items with a probabilistic method and conducts probabilistic logical reasoning on users' interaction patterns. Then the feature and logic representations learned from the DNN and logic network are concatenated to make the prediction. Finally, experiments on various sequential recommendation models demonstrate the effectiveness of the SR-PLR. Our code is available at https://github.com/Huanhuaneryuan/SR-PLR.

The security of smart contracts has garnered considerable attention given the potential for substantial financial losses and erosion of trust in blockchain platforms. Numerous methods have been proposed to detect vulnerabilities in smart contracts. Notably, as the number of smart contracts continues to proliferate, automated techniques based on deep learning (DL) are making remarkable progress. However, a significant challenge persists in acquiring an efficient embedding representation that is compatible with DL models with input length restrictions. In this paper, we propose a novel detection method named GraBit for identifying reentrancy vulnerability-one of the most critical vulnerabilities in smart contracts. GraBit leverages the pre-trained model GraphCodeBERT to embed both the source code and concise key data flow graphs extracted from the code. Additionally, we customize a sequential model based on Bi-directional Long Short-Term Memory and attention mechanism to effectively capture contextual semantic information. To evaluate the performance of GraBit, we conduct extensive experiments on a public large-scale dataset. Our experimental results reveal that GraBit achieves a remarkable F1-score of 94.44% in detecting reentrancy vulnerability, outperforming state-of-the-art methods.

The self-attention mechanism, which equips with a strong capability of modeling long-range dependencies, is one of the extensively used techniques in the sequential recommendation field. However, many recent studies represent that current self-attention based models are low-pass filters and are inadequate to capture high-frequency information. Furthermore, since the items in the user behaviors are intertwined with each other, these models are incomplete to distinguish the inherent periodicity obscured in the time domain. In this work, we shift the perspective to the frequency domain, and propose a novel Frequency Enhanced Hybrid Attention Network for Sequential Recommendation, namely FEARec. In this model, we firstly improve the original time domain self-attention in the frequency domain with a ramp structure to make both low-frequency and high-frequency information could be explicitly learned in our approach. Moreover, we additionally design a similar attention mechanism via auto-correlation in the frequency domain to capture the periodic characteristics and fuse the time and frequency level attention in a union model. Finally, both contrastive learning and frequency regularization are utilized to ensure that multiple views are aligned in both the time domain and frequency domain. Extensive experiments conducted on four widely used benchmark datasets demonstrate that the proposed model performs significantly better than the state-of-the-art approaches.

Sequential recommendation aims to capture users' dynamic interest and predicts the next item of users' preference. Most sequential recommendation methods use a deep neural network as sequence encoder to generate user and item representations. Existing works mainly center upon designing a stronger sequence encoder. However, few attempts have been made with training an ensemble of networks as sequence encoders, which is more powerful than a single network because an ensemble of parallel networks can yield diverse prediction results and hence better accuracy. In this paper, we present Ensemble Modeling with contrastive Knowledge Distillation for sequential recommendation (EMKD). Our framework adopts multiple parallel networks as an ensemble of sequence encoders and recommends items based on the output distributions of all these networks. To facilitate knowledge transfer between parallel networks, we propose a novel contrastive knowledge distillation approach, which performs knowledge transfer from the representation level via Intra-network Contrastive Learning (ICL) and Cross-network Contrastive Learning (CCL), as well as Knowledge Distillation (KD) from the logits level via minimizing the Kullback-Leibler divergence between the output distributions of the teacher network and the student network. To leverage contextual information, we train the primary masked item prediction task alongside the auxiliary attribute prediction task as a multi-task learning scheme. Extensive experiments on public benchmark datasets show that EMKD achieves a significant improvement compared with the state-of-the-art methods. Besides, we demonstrate that our ensemble method is a generalized approach that can also improve the performance of other sequential recommenders. Our code is available at this link: https://github.com/hw-du/EMKD.

Contrastive Learning (CL) performances as a rising approach to address the challenge of sparse and noisy recommendation data. Although having achieved promising results, most existing CL methods only perform either hand-crafted data or model augmentation for generating contrastive pairs to find a proper augmentation operation for different datasets, which makes the model hard to generalize. Additionally, since insufficient input data may lead the encoder to learn collapsed embeddings, these CL methods expect a relatively large number of training data (e.g., large batch size or memory bank) to contrast. However, not all contrastive pairs are always informative and discriminative enough for the training processing. Therefore, a more general CL-based recommendation model called Meta-optimized Contrastive Learning for sequential Recommendation (MCLRec) is proposed in this work. By applying both data augmentation and learnable model augmentation operations, this work innovates the standard CL framework by contrasting data and model augmented views for adaptively capturing the informative features hidden in stochastic data augmentation. Moreover, MCLRec utilizes a meta-learning manner to guide the updating of the model augmenters, which helps to improve the quality of contrastive pairs without enlarging the amount of input data. Finally, a contrastive regularization term is considered to encourage the augmentation model to generate more informative augmented views and avoid too similar contrastive pairs within the meta updating. The experimental results on commonly used datasets validate the effectiveness of MCLRec.

Radar echo extrapolation based on deep learning is an important method for conducting precipitation nowcasting. Radar echo sequence data have spatio-temporal correlations and non-rigid movements of the radar echo. According to the characteristics of radar data, this study proposes a new spatio-temporal fusion neural network called STUNNER. STUNNER implements a two-stream spatio-temporal fusion strategy to extract and fuse spatial and temporal signals. Specifically, it uses a novel cross-network embedding method to achieve efficient spatio-temporal fusion; the fusion integrates a temporal differencing network and a spatio-temporal trajectory network. The temporal differencing network models the high-order non-stationarity of the radar sequence to learn the motion trend. The spatio-temporal trajectory network optimizes the convolution operator in a spatio-temporal long short-term memory model to extract the transient variations from the radar images. We compare STUNNER with other five models on two public datasets. On the radar echo extrapolation task, STUNNER achieves the best performance.

Sequential recommendation has been a widely popular topic of recommender systems. Existing works have contributed to enhancing the prediction ability of sequential recommendation systems based on various methods, such as recurrent networks and self-attention mechanisms. However, they fail to discover and distinguish various relationships between items, which could be underlying factors which motivate user behaviors. In this article, we propose an Edge-Enhanced Global Disentangled Graph Neural Network (EGD-GNN) model to capture the relation information between items for global item representation and local user intention learning. At the global level, we build a global-link graph over all sequences to model item relationships. Then a channel-aware disentangled learning layer is designed to decompose edge information into different channels, which can be aggregated to represent the target item from its neighbors. At the local level, we apply a variational auto-encoder framework to learn user intention over the current sequence. We evaluate our proposed method on three real-world datasets. Experimental results show that our model can get a crucial improvement over state-of-the-art baselines and is able to distinguish item features.

Exploiting social networks is expected to enhance the performance of recommender systems when interaction information is sparse. Existing social recommendation models focus on modeling multi-graph structures and then aggregating the information from these multiple graphs to learn potential user preferences. However, these methods often employ complex models and redundant parameters to get a slight performance improvement. Contrastive learning has been widely researched as an effective paradigm in the area of recommendation. Most existing contrastive learning-based models usually focus on constructing multi-graph structures to perform graph augmentation for contrastive learning. However, the effect of graph augmentation on contrastive learning is inconclusive. In view of these challenges, in this work, we propose a contrastive learning based graph convolution network for social recommendation (CLSR), which integrates information from both the social graph and the interaction graph. First, we propose a fusion-simplified method to combine the social graph and the interaction graph. Technically, on the basis of exploring users’ interests by interaction graph, we further exploit social connections to alleviate data sparsity. By combining the user embeddings learned through two graphs in a certain proportion, we can obtain user representation at a finer granularity. Meanwhile, we introduce a contrastive learning framework for multi-graph network modeling, where we explore the feasibility of constructing positive and negative samples of contrastive learning by conducting data augmentation on embedding representations. Extensive experiments verify the superiority of CLSR’s contrastive learning framework and fusion-simplified method of integrating social relations.

Sequential recommendation, which aims to recommend next item that the user will likely interact in a near future, has become essential in various Internet applications. Existing methods usually consider the transition patterns between items, but ignore the transition patterns between features of items. We argue that only the item-level sequences cannot reveal the full sequential patterns, while explicit and implicit feature-level sequences can help extract the full sequential patterns. Meanwhile, the item-level sequential recommendation also suffers from limited supervised signal issues. In this article, we propose a novel model Feature-level Deeper Self-Attention Network with Contrastive Learning (FDSA-CL) for sequential recommendation. Specifically, FDSA-CL first integrates various heterogeneous features of items into feature-level sequences with different weights through a vanilla attention mechanism. After that, FDSA-CL applies separated self-attention blocks on item-level sequences and feature-level sequences, respectively, to model item transition patterns and feature transition patterns. Moreover, we propose contrastive learning and item feature recommendation tasks to capture the embedding commonality and further utilize the beneficial interaction among the two levels, so as to alleviate the sparsity of the supervised signal and extract the most critical information. Finally, we jointly optimize the above tasks. We evaluate the proposed model using two real-world datasets and experimental results show that our model significantly outperforms the state-of-the-art approaches.

With the explosive growth of short texts on users' preferences, learning discriminative and coherent latent topics from short texts is a critical work, since many practical applications require semantic understandings that short texts convey explicitly and implicitly. However, existing short text topic learning methods face the challenge of fully capturing semantically related co-occurrence phrases. Therefore, this paper proposes a novel Heterogeneous Information Network-based Short Text Topic learning approach (HIN-ShoTT) in terms of parts of speech, without depending on any auxiliary information. Specifically, HIN-ShoTT can be decomposed into three phases: i) seeking semantic relations among words, where HIN-ShoTT models multiple semantic relations among words based on a Heterogeneous Information Network (HIN) in terms of parts of speech; ii) extracting co-occurrence phrases and filtering noises, where HIN-ShoTT defines parts-of-speech meta structures to guide co-occurrence phrase extraction and a self-adapting threshold filtering module is proposed for discarding noises; and iii) inferring topics, where HIN-ShoTT models the generative process of co-occurrence phrases to make topic learning effective with the abundant corpus-level information. Our experimental results on three real-world datasets not only show that HIN-ShoTT performs well, but also demonstrate that it is feasible to incorporate HIN into short text topic learning for accuracy improvement.

Effective streaming feature selection in dynamic on-line environments is essential in numerous applications. However, most existing methods evaluate high-dimensional features individually and ignore the potentially pertainable group structures of features. Moreover, the class imbalance underlying streaming data may further decrease the discriminative efficacy of the selected features, resulting in deteriorated classification performance. Motivated by this observation, we propose a proximal cost-sensitive sparse group online learning (PCSGOL) framework to handle imbalanced and high-dimensional streaming data. Specifically, we formulate this issue as a new cost-sensitive online optimization problem by leveraging the ℓ2-norm, ℓ1-norm, and group-wise sparsity constraints in the dual averaging regularization. The average weighted distance is also introduced in PCSGOL to achieve stable prediction results. We mathematically derive closed-form solutions to the optimization problems with four modified hinge loss functions, leading to four variants of PCSGOL. Extensive empirical studies on real-world streaming datasets demonstrate the effectiveness of our proposed method.

Contrastive learning with Transformer-based sequence encoder has gained predominance for sequential recommendation. It maximizes the agreements between paired sequence augmentations that share similar semantics. However, existing contrastive learning approaches in sequential recommendation mainly center upon left-to-right unidirectional Transformers as base encoders, which are suboptimal for sequential recommendation because user behaviors may not be a rigid left-to-right sequence. To tackle that, we propose a novel framework named Contrastive learning with Bidirectional Transformers for sequential recommendation (CBiT). Specifically, we first apply the slide window technique for long user sequences in bidirectional Transformers, which allows for a more fine-grained division of user sequences. Then we combine the cloze task mask and the dropout mask to generate high-quality positive samples and perform multi-pair contrastive learning, which demonstrates better performance and adaptability compared with the normal one-pair contrastive learning. Moreover, we introduce a novel dynamic loss reweighting strategy to balance between the cloze task loss and the contrastive loss. Experiment results on three public benchmark datasets show that our model outperforms state-of-the-art models for sequential recommendation. Our code is available at this link: https://github.com/hw-du/CBiT/tree/master.

Most existing online learning methods focus on mining ever-evolving streaming data based on the principle of first-order optimization. However, one drawback of these methods is the slow convergence rate in each iteration, resulting in sub-optimal solutions and deteriorated performance. Second-order methods, while are able to provide faster convergence, have been under-studied due to the high cost of computing the curvature information. To address this problem, in this paper, we develop a second-order projection dual averaging based online learning (SPDA) method to effectively handle high-throughput streaming data. By fully exploiting the regularized dual averaging optimization, the second-order information, and an optimal projection operator, SPDA converges fast with fairly optimal solutions. Two speed-up versions of SPDA, i.e., SPDA-diag and SPDA-sketch, are developed via the diagonal operator and Hessian sketch, respectively. Theoretical derivations on the regret bound of SPDA establish a solid convergence guarantee for this method. Extensive experiments demonstrate the efficacy of the proposed algorithms on large-scale online learning tasks, such as online binary and multi-class classification and online anomaly detection, shedding light on their potential wide applications.

Graph Convolution Network (GCN) has been applied in recommendation with various architectures for its representation learning capability in graph-structured data. Despite existing GCN-based recommendation models successfully capturing the user-item interactions, they still suffer from two limitations. On the one hand, they model users and items in the Euclidean space with real-value embeddings, which have high distortion when modeling complex graphs. On the other hand, they have not fully explored contrastive learning for GCN-based recommendations. Simply applying augmentation pairs of the same type may make features less generalizable and lead to sub-optimal performance. To this end, in this paper, we propose a Quaternion-based Graph Contrastive Learning (QGCL) recommendation model. It embeds all users and items into the Quaternion space and performs message propagation with quaternion graph convolution layers. Moreover, we attempt to compose different types of data augmentations for augmented views in graph contrastive learning as an auxiliary task. We evaluate the proposed model using three public datasets, and experimental results demonstrate significant improvements over the state-of-the-art methods by a large margin.

Recently, contrastive learning alleviates data sparsity issues and improves the performance of the Graph Neural Network (GNN) recommender models by employing graph structure dropout augmentations. However, these models still face following limitations: (1) Information loss. Dropout may discard helpful information. (2) Insufficient utilization of path-level information. Meta-path is carried numerous high-order information, which has not been well considered in these models. To this end, in this paper, we propose a novel framework, Node and Meta-Path Contrastive Learning for Recommender Systems (NPCRS), which utilizes meta-path to capture path-level information for model learning. Specifically, our approach first generates a meta-path view on the user-item bipartite graph by leveraging meta-path instead of random dropout. Then, we learn the node representation on a user-item bipartite graph and meta-path view to capture both node and path-level information simultaneously. Further, a multi-positive sample mechanism is introduced to define positive and negative samples for contrastive learning. Finally, NPCRS utilizes contrastive learning to learn a more informative node representation. We evaluate the proposed model using three real-world datasets and our experimental results show that our model significantly outperforms the state-of-the-art approaches.

Nowadays billions of smart objects are connected to the internet and interact with the cloud. Remote monitoring, control systems, and data analysis becomes more intelligent with the huge amount of data been collecting and crowdsourcing. To reduce data transmission and improve executive efficiency, computing and storing functions are executed toward edge devices. However, the great convenience raises numerous issues including reliability and security of the sensors and the control systems because these issues have not always been considered the top priority. Accordingly, many new research opportunities and challenges for intelligent sensing and control have arisen.

Crowdsourcing is a hotspot research field which can facilitate machine learning by collecting labels to train models. Consequently, the state-of-the-art research efforts in crowdsourcing focus on truth inference or label integration, to remove inconsistent labels or to alleviate biased labeling. In turn, the integrated labels will be used to fine-tune machine learning models. Particularly, in this paper, we change the target of truth inference in crowdsourcing from discrete labels to multiple comments given by online participants, that is, the integration of the crowdsourced comments. For such a goal, we propose aSelf-play andSentiment-EmphasizedCommentIntegrationFramework (SSECIF), based on deepQ-learning, with three unique features. First, our framework SSECIF can generate the comment integration in a totally self-play way, without relying on the ground truth generated by human effort. Second, the integrated comment generated by SSECIF can include salient content with low redundancy. Third, the proposed framework SSECIF has emphasized, with a higher intensity, the sentiment in the integrated comment, in order to reflect the attitude or opinion more obviously. Extensive evaluation on real-world datasets demonstrates that SSECIF has achieved the best overall performance in terms of both effectiveness and efficiency, compared with the state-of-the-art methods.

Next Point-of-Interest (POI) recommendation which is of great value to both users and POI holders is a challenging task since complex sequential patterns and rich contexts are contained in extremely sparse user check-in data. Recently proposed embedding techniques have shown promising results in alleviating the data sparsity issue by modeling context information, and Recurrent Neural Network (RNN) has been proved effective in the sequential prediction. However, existing next POI recommendation approaches train the embedding and network model separately, which cannot fully leverage rich contexts. In this paper, we propose a novel unified neural network framework, named NeuNext, which leverages POI context prediction to assist next POI recommendation by joint learning. Specifically, the Spatio-Temporal Gated Network (STGN) is proposed to model personalized sequential patterns for users’ long and short term preferences in the next POI recommendation. In the POI context prediction, rich contexts on POI sides are used to construct graph, and enforce the smoothness among neighboring POIs. Finally, we jointly train the POI context prediction and the next POI recommendation to fully leverage labeled and unlabeled data. Extensive experiments on real-world datasets show that our method outperforms other approaches for next POI recommendation in terms of Accuracy and MAP.

Android is a growing target for malicious software (malware) because of its popularity and functionality. Malware poses a serious threat to users’ privacy, money, equipment and file integrity. A series of data-driven malware detection methods were proposed. However, there exist two key challenges for these methods: (1) how to learn effective feature representation from raw data; (2) how to reduce the dependence on the prior knowledge or human labors in feature learning. Inspired by the success of deep learning methods in the feature representation learning community, we propose a malware detection framework which starts with learning rich-features by a novel unsupervised feature learning algorithm Merged Sparse Auto-Encoder (MSAE). In order to extract more compact and discriminative feature from the rich-features to further boost the malware detection capability, a hybrid deep network learning algorithm Stacked Hybrid Learning MSAE and SDAE (SHLMD) is established by further incorporating a classical deep learning method Stacked Denoising Auto-encoders (SDAE). After that, we feed the feature learned by MSAE and SHLMD respectively to classification algorithms, e.g., Support Vector Machine (SVM) or K-NearestNeighbor (KNN), to train a malware detection model. Evaluation results on two real-world datasets demonstrate that SHLMD achieves 94.46 and 90.57 percent accuracy respectively, which outperforms the classical unsupervised feature representation learning Sparse Auto-encoder (SAE). MSAE performs similarly to SAE. SHLMD can further improve the performance of MSAE and the supervised fine-tuned method SDAE. Besides, we compare the performance of our methods with that of state-of-the-art detection approaches, including classical deep-learning-based methods. Extensive experiments show that our proposed methods are effective enough to detect Android malware.

In wireless sensor network (WSN), the energy of sensor nodes is limited. Designing efficient routing method for reducing energy consumption and extending the WSN’s lifetime is important. This paper proposes a novel energy‐efficient, static scenario‐oriented routing method of WSN based on edge computing named the NEER, in which WSN is divided into several areas according to the coverage of gateway (or base station), and each of the areas is regarded as an edge area network (EAN). Each edge area network is abstracted into a weighted undirected graph model combined with the residual energy of the sensor nodes. The base station (or a gateway) calculates the optimal energy consumption path for all sensor nodes within its coverage, and the nodes then perform data transmission through their suggested optimal paths. The proposed method is verified by the simulations, and the results show that the proposed method may consume about 37% less energy compared with the conventional WSN routing protocol and can also effectively extend the lifetime of WSN.

Integrating multiple comments into a concise statement for any online products or web services requires a non-trivial understanding of the input. Recently, graph neural networks (GNN) has been successfully applied to learn from highly-structured graph representations to mitigate the relationship between entities, such as co-references. However, current inter-sentence relation extraction cannot leverage discrete reasoning chains over multiple comments. To address this issue, in this paper, we propose a probabilistic $K$-hop knowledge graph (KKG) to extend existing knowledge graphs with inferred relations via discrete intra-sentence and inter-sentence reasoning chains. KKG associates each inferred relation with a confidence value through Bayesian inference. We further answer how a knowledge graph with inferred relations can help the multiple comments integration through integrating KKG with GNN ($\text{K}^2$-GNN). Our extensive experimental results show that our $\text{K}^2$-GNN outperforms all baseline graph models on multiple comments integration.

For decades, research in natural language processing (NLP) has focused on summarization. Sequence-to-sequence models for abstractive summarization have been studied extensively, yet generated summaries commonly suffer from fabricated content, and are often found to be near-extractive. We argue that, to address these issues, summarizers need to acquire the co-references that form multiple types of relations over input sentences, e.g., 1-to-N, N-to-1, and N-to-N relations, since the structured knowledge for text usually appears on these relations. By allowing the decoder to pay different attention to the input sentences for the same entity at different generation states, the structured graph representations generate more informative summaries. In this paper, we propose a hierarchical graph attention networks (HGATs) for abstractive summarization with a topic-sensitive PageRank augmented graph. Specifically, we utilize dual decoders, a sequential sentence decoder, and a graph-structured decoder (which are built hierarchically) to maintain the global context and local characteristics of entities, complementing each other. We further design a greedy heuristic to extract salient users' comments while avoiding redundancy to drive a model to better capture entity interactions. Our experimental results show that our models produce significantly higher ROUGE scores than variants without graph-based attention on both SSECIF and CNN/Daily Mail (CNN/DM) datasets.

Private information can either take the form of key phrases that are explicitly contained in the text or be implicit. For example, demographic information about the author of a text can be predicted with above-chance accuracy from linguistic cues in the text itself. Letting alone its explicitness, some of the private information correlates with the output labels and therefore can be learned by a neural network. In such a case, there is a tradeoff between the utility of the representation (measured by the accuracy of the classification network) and its privacy. This problem is inherently a multi-objective problem because these two objectives may conflict, necessitating a trade-off. Thus, we explicitly cast this problem as multi-objective optimization (MOO) with the overall objective of finding a Pareto stationary solution. We, therefore, propose a multiple-gradient descent algorithm (MGDA) that enables the efficient application of the Frank-Wolfe algorithm [10] using the line search. Experimental results on sentiment analysis and part-of-speech (POS) tagging show that MGDA produces higher-performing models than most recent proxy objective approaches, and performs as well as single objective baselines.

Sequential recommendation has become an attractive topic in recommender systems. Existing sequential recommendation methods, including the methods based on the state-of-the-art self-attention mechanism, usually employ deterministic neural networks to represent user preferences as fixed-points in the latent feature spaces. However, the fixed-point vector lacks the ability to capture the uncertainty and dynamics of user preferences that are prevalent in recommender systems. In this paper, we propose a new Variational Self-Attention Network (VSAN), which introduces a variational autoencoder (VAE) into the self-attention network to capture latent user preferences. Specifically, we represent the obtained self-attention vector as density via variational inference, whose variance well characterizes the uncertainty of user preferences. Furthermore, we employ self-attention networks to learn the inference process and generative process of VAE, which well captures long-range and local dependencies. Finally, we evaluate our proposed method VSAN with two public real-world datasets. Our experimental results show the effectiveness of our model compared to the state-of-the-art approaches.

Recent studies have shown that graph learning models are highly vulnerable to adversarial attacks, and network alignment methods are no exception. How to enhance the robustness of network alignment against adversarial attacks remains an open research problem. In this paper, we propose a robust network alignment solution, RNA, for offering preemptive protection of existing network alignment algorithms, enhanced with the guidance of effective adversarial attacks. First, we analyze how popular iterative gradient-based adversarial attack techniques suffer from gradient vanishing issues and show a fake sense of attack effectiveness. Based on dynamical isometry theory, an attack signal scaling (ASS) method with established upper bound of feasible signal scaling is introduced to alleviate the gradient vanishing issues for effective adversarial attacks while maintaining the decision boundary of network alignment. Second, we develop an adversarial perturbation elimination (APE) model to neutralize adversarial nodes in vulnerable space to adversarial-free nodes in safe area, by integrating Dirac delta approximation (DDA) techniques and the LSTM models. Our proposed APE method is able to provide proactive protection to existing network alignment algorithms against adversarial attacks. The theoretical analysis demonstrates the existence of an optimal distribution for the APE model to reach a lower bound. Last but not least, extensive evaluation on real datasets presents that RNA is able to offer the preemptive protection to trained network alignment methods against three popular adversarial attack models.

With the ever-increasing maturity and popularization of the Internet of Things (IoT), tremendous business applications developed by distinct enterprises or organizations have been encapsulated into lightweight web services that can easily be accessed or invoked remotely. However, the big volume of candidate web services places a heavy burden on the users' service selection decision-making process. Under the circumstance, a variety of intelligent recommendation solutions have been developed to reduce the high decision-making cost. Traditional resolutions usually challenge in two aspects. First, the recommendation parameters, i.e., the quality of services (QoS), usually relies on user/service location heavily; therefore, low-quality recommended results may be returned to users if user/service location information is overlooked. Second, historical QoS data often contain partial sensitive information of users; therefore, it becomes a necessity to protect the sensitive QoS data while making accurate recommendation decisions. To tackle the above challenges, we introduce the concepts of user/service location information and locality-sensitive hashing (LSH) in the domain and propose a location-aware recommendation approach with privacy-preservation capability. A wide range of experiments is set up based on the popular WS-DREAM data set, whose results prove the effectiveness and efficiency of our approach.

In digitalized and automated systems, more and more intelligent devices have become an import part of industrial Internet of Things (IIOT). However, the lack of security in IIOT makes people facing unprecedented threats from the Dark web. Traffic classification is an important means to prevent anonymous attacks. However, the growing usage of smartphones in daily life is deeply changing the nature of network traffic, which makes traffic classification more challenging. In this article, we propose a Tor traffic identification and multilevel classification framework based on network flow features, which realizes the identification of anonymous traffic (L1), traffic types (L2) of anonymous traffic, and applications (L3) on a mobile and a PC platform, respectively. We further analyze differences between the mobile and the PC platform. We conclude that the impact of time-related features is higher than that of the nontime-related features on the mobile platform, while it is opposite on the PC platform. And it is more difficult to identify and classify Tor types (L2) and specific Tor applications (L3) on the mobile platform than on the PC platform, including using different number of features and early identification and classification.

Recently accumulated massive amounts of geo-tagged photos provide an excellent opportunity to understand human behaviors and can be used for personalized tour recommendation. However, no existing work has considered the visual content information in these photos for tour recommendation. We believe the visual features of photos provide valuable information on measuring user / Point-of-Interest (POI) similarities, which is challenging due to data sparsity. To this end, in this paper, we propose a visual feature enhanced tour recommender system, named ‘Photo2Trip’, to utilize the visual contents and collaborative filtering models for recommendation. Specifically, we propose a Visual-enhanced Probabilistic Matrix Factorization model (VPMF), which integrates visual features into the collaborative filtering model, to learn user interests by leveraging the historical travel records. We then extend VPMF to End-to-End training framework to incorporate users (POIs) latent factors into the learning process of the visual content of photos, which generalizes the applicability of the proposed VPMF framework in tour recommendation. Extensive empirical studies verify that our proposed visual-enhanced personalized tour recommendation method outperforms other benchmark methods in terms of recommendation accuracy. The results also show that visual features are effective in alleviating the data sparsity and cold start problems on personalized tour recommendation.

Learning from crowds often performs in an active learning paradigm, aiming to improve learning performance quickly as well as to reduce labeling cost by selecting proper workers to (re)label critical instances. Previous active learning methods for learning from crowds do not have any proactive mechanism to effectively improve the reliability of workers, which prevents to obtain steadily rising learning curves. To help workers improve their reliability while performing tasks, this paper proposes a novel Interactive Learning framework with Proactive Cognitive Enhancement (ILPCE) for crowd workers. The ILPCE framework includes an interactive learning mechanism: When crowd workers perform labeling tasks in active learning, their cognitive ability to the specific domain can be enhanced through learning the exemplars selected by a psychological model-based machine teaching method. A novel probabilistic truth inference model and an interactive labeling scheme are proposed to ensure the effectiveness of the interactive learning mechanism and the performance of learning models can be simultaneously improved through a fast and low-cost way. Experimental results on three real-world learning tasks demonstrate that our ILPCE significantly outperforms five representative state-of-the-art methods.

Chronic alcohol consumption has become one of the major concerns of the society in the sector of public health. The death toll due to excessive consumption of alcohol is increasing exponentially. Face-to-face interaction to create awareness and to stymie the consumption of alcohol is a quixotic solution. Alcohol relapse remains a challenging problem in disorders associated with alcohol addiction which is related to spatial-temporal factors like periods and specific places. In this paper, a new generation of relapse prevention mobile application called “BeSober” is proposed. It assists users to develop abstemious habits in drinking, providing support within the intervention period, tracking risky alcoholic spots, presenting community-based support from alternative users or alcohol addiction therapists, monitoring addict's behaviour and offer personalized recommendations to assist the addict in staying sober.

Session-based recommendation becomes a research hotspot for its ability to make recommendations for anonymous users. However, existing session-based methods have the following limitations: (1) They either lack the capability to learn complex dependencies or focus mostly on the current session without explicitly considering collaborative information. (2) They assume that the representation of an item is static and fixed for all users at each time step. We argue that even the same item can be represented differently for different users at the same time step. To this end, we propose a novel solution, Collaborative Self-Attention Network (CoSAN) for session-based recommendation, to learn the session representation and predict the intent of the current session by investigating neighborhood sessions. Specially, we first devise a collaborative item representation by aggregating the embedding of neighborhood sessions retrieved according to each item in the current session. Then, we apply self-attention to learn long-range dependencies between collaborative items and generate collaborative session representation. Finally, each session is represented by concatenating the collaborative session representation and the embedding of the current session. Extensive experiments on two real-world datasets show that CoSAN constantly outperforms state-of-the-art methods.

Multi-label classification (MLC) is pervasive in real-world applications. Conventional MLC algorithms assume that enough ground truth labels are available for training a classifier. While in reality, obtaining ground truth labels is expensive and time-consuming. In the field of data mining, it is more efficient to use crowdsourcing for label collection. In this setting, an MLC algorithm needs to deal with the noisiness of the crowdsourced labels as well as the remaining massive unlabeled data. In this paper, we propose a deep generative model to describe the label generation process for this semi-supervised multi-label learning problem. Although deep generative models are widely used for MLC problems, no previous work could address the noisy crowdsourced multi-labels and unlabeled data simultaneously. To address this challenging problem, our novel generative model incorporates latent variables to describe the labeled/unlabeled data as well as the labeling process of crowdsourcing. We introduce an efficient sequential inference model to approximate the model posterior and infer the ground truth labels. Our experimental results on various scales of datasets demonstrate the effectiveness of our proposed model. It performs favorably against four state-of-the-art deep generative models.

Visual aesthetics of products plays an important role in the decision process when purchasing appearance-first products, e.g., clothes. Indeed, user’s aesthetic preference, which serves as a personality trait and a basic requirement, is domain independent and could be used as a bridge between domains for knowledge transfer. However, existing work has rarely considered the aesthetic information in product images for cross-domain recommendation. To this end, in this paper, we propose a new deep Aesthetic Cross-Domain Networks (ACDN), in which parameters characterizing personal aesthetic preferences are shared across networks to transfer knowledge between domains. Specifically, we first leverage an aesthetic network to extract aesthetic features. Then, we integrate these features into a cross-domain network to transfer users’ domain independent aesthetic preferences. Moreover, network cross-connections are introduced to enable dual knowledge transfer across domains. Finally, the experimental results on real-world datasets show that our proposed model ACDN outperforms benchmark methods in terms of recommendation accuracy.

With crowdsourcing systems, labels can be obtained with low cost, which facilitates the creation of training sets for prediction model learning. However, the labels obtained from crowdsourcing are often imperfect, which brings great challenges in model learning. Since 2008, the machine learning community has noticed the great opportunities brought by crowdsourcing and has developed a large number of techniques to deal with inaccuracy, randomness, and uncertainty issues when learning with crowdsourcing. This paper summarizes the technical progress in this field during past eleven years. We focus on two fundamental issues: the data (label) quality and the prediction model quality. For data quality, we summarize ground truth inference methods and some machine learning based methods to further improve data quality. For the prediction model quality, we summarize several learning paradigms developed under the crowdsourcing scenario. Finally, we further discuss several promising future research directions to attract researchers to make contributions in crowdsourcing.

Next Point-of-Interest (POI) recommendation is of great value for both location-based service providers and users. However, the state-of-the-art Recurrent Neural Networks (RNNs) rarely consider the spatio-temporal intervals between neighbor check-ins, which are essential for modeling user check-in behaviors in next POI recommendation. To this end, in this paper, we propose a new Spatio-Temporal Gated Network (STGN) by enhancing long-short term memory network, where spatio-temporal gates are introduced to capture the spatio-temporal relationships between successive checkins. Specifically, two pairs of time gate and distance gate are designed to control the short-term interest and the longterm interest updates, respectively. Moreover, we introduce coupled input and forget gates to reduce the number of parameters and further improve efficiency. Finally, we evaluate the proposed model using four real-world datasets from various location-based social networks. The experimental results show that our model significantly outperforms the state-ofthe-art approaches for next POI recommendation.

Session-based recommendation, which aims to predict the user's immediate next action based on anonymous sessions, is a key task in many online services (e.g., e-commerce, media streaming). Recently, Self-Attention Network (SAN) has achieved significant success in various sequence modeling tasks without using either recurrent or convolutional network. However, SAN lacks local dependencies that exist over adjacent items and limits its capacity for learning contextualized representations of items in sequences. In this paper, we propose a graph contextualized self-attention model (GC-SAN), which utilizes both graph neural network and self-attention mechanism, for session-based recommendation. In GC-SAN, we dynamically construct a graph structure for session sequences and capture rich local dependencies via graph neural network (GNN). Then each session learns long-range dependencies by applying the self-attention mechanism. Finally, each session is represented as a linear combination of the global preference and the current interest of that session. Extensive experiments on two real-world datasets show that GC-SAN outperforms state-of-the-art methods consistently.

Sequential recommendation, which aims to recommend next item that the user will likely interact in a near future, has become essential in various Internet applications. Existing methods usually consider the transition patterns between items, but ignore the transition patterns between features of items. We argue that only the item-level sequences cannot reveal the full sequential patterns, while explicit and implicit feature-level sequences can help extract the full sequential patterns. In this paper, we propose a novel method named Feature-level Deeper Self-Attention Network (FDSA) for sequential recommendation. Specifically, FDSA first integrates various heterogeneous features of items into feature sequences with different weights through a vanilla mechanism. After that, FDSA applies separated self-attention blocks on item-level sequences and feature-level sequences, respectively, to model item transition patterns and feature transition patterns. Then, we integrate the outputs of these two blocks to a fully-connected layer for next item recommendation. Finally, comprehensive experimental results demonstrate that considering the transition relationships between features can significantly improve the performance of sequential recommendation.

The sequential recommendation, which models sequential behavioral patterns among users for the recommendation, plays a critical role in recommender systems. However, the state-of-the-art Recurrent Neural Networks (RNN) solutions rarely consider the non-linear feature interactions and non-monotone short-term sequential patterns, which are essential for user behavior modeling in sparse sequence data. In this paper, we propose a novel Recurrent Convolutional Neural Network model (RCNN). It not only utilizes the recurrent architecture of RNN to capture complex long-term dependencies, but also leverages the convolutional operation of Convolutional Neural Network (CNN) model to extract short-term sequential patterns among recurrent hidden states. Specifically, we first generate a hidden state at each time step with the recurrent layer. Then the recent hidden states are regarded as an “image”, and RCNN searches non-linear feature interactions and non-monotone local patterns via intra-step horizontal and inter-step vertical convolutional filters, respectively. Moreover, the output of convolutional filters and the hidden state are concatenated and fed into a fully-connected layer to generate the recommendation. Finally, we evaluate the proposed model using four real-world datasets from various application scenarios. The experimental results show that our model RCNN significantly outperforms the state-of-the-art approaches on sequential recommendation.

Correlation Degree Optimization Driven Random Forest for Detecting DDoS Attacks in Cloud Computing" [1], there was an error in the expression below formula (1) in Section 3.2, where theta "" symbol should be replaced with alpha "".Therefore the expression "() = () + (1 -)(), (0 < < 1)" should be corrected to be "() = () + (1 -)(), (0 < < 1)".

Complex Event Processing (CEP) systems extract interest situations from event flows based on event detection patterns. However, local event processing for distributed Internet of Things (IoT) has not been discussed yet. Besides, it is complex or impossible to discover such patterns in some applications of IoT. In this article, we design a complex event service to process event flows based on IoT resource models, which does not depend on existing patterns, and deals with both discrete events and continuous variables. To improve the CEP performance, local IoT resources are used for local event processing, and a lazy exchange method is designed to realize the collaborated event processing between network edges and a data center. Our evaluation shows that our solution is feasible and effective.

With the crowdsourcing of small tasks becoming easier, it is possible to obtain non-expert/imperfect labels at low cost. With low-cost imperfect labeling, it is straightforward to collect multiple labels for the same data items. This paper proposes strategies of utilizing these multiple labels for supervised learning, based on two basic ideas: majority voting and pairing. We show several interesting results based on our experiments. (i) The strategies based on the majority voting idea work well under the situation where the certainty level is high. (ii) On the contrary, the pairing strategies are more preferable under the situation where the certainty level is low. (iii) Among the majority voting strategies, soft majority voting can reduce the bias and roughness, and perform better than majority voting. (iv) Pairing can completely avoid the bias by having both sides (potentially correct and incorrect/noisy information) considered. Beta estimation is applied to reduce the impact of the noise in pairing. Our experimental results show that pairing with Beta estimation always performs well under different certainty levels. (v) All strategies investigated are labeling quality agnostic strategies for real-world applications, and some of them perform better than or at least very close to the gnostic strategies.

Traditional learning from crowdsourced labeled data consists of two stages: inferring true labels for instances from their multiple noisy labels and building a learning model using these instances with the inferred labels. This straightforward two-stage learning scheme suffers from two weaknesses: (1) the accuracy of inference may be very low; (2) useful information may be lost during inference. In this paper, we proposed a novel ensemble method for learning from crowds. Our proposed method is a meta-learning scheme. It first uses a bootstrapping process to create M sub-datasets from an original crowdsourced labeled dataset. For each sub-dataset, each instance is duplicated with different weights according to the distribution and class memberships of its multiple noisy labels. A base classifier is then trained from this extended sub-dataset. Finally, unlabeled instances are predicted by aggregating the outputs of these M base classifiers. Because the proposed method gets rid of the inference procedure and uses the full dataset to train learning models, it preserves the useful information for learning as much as possible. Experimental results on nine simulated and two real-world crowdsourcing datasets consistently show that the proposed ensemble learning method significantly outperforms five state-of-the-art methods.

With online crowdsourcing platforms, labels can be acquired at relatively low costs from massive nonexpert workers. To improve the quality of labels obtained from these imperfect crowdsourced workers, we usually let different workers provide labels for the same instance. Then, the true labels for all instances are estimated from these multiple noisy labels. This traditional general-purpose label aggregation process, solely relying on the collected noisy labels, cannot significantly improve the accuracy of integrated labels under a low labeling quality circumstance. This paper proposes a novel bilayer collaborative clustering (BLCC) method for the label aggregation in crowdsourcing. BLCC first generates the conceptual-level features for the instances from their multiple noisy labels and infers the initially integrated labels by performing clustering on the conceptual-level features. Then, it performs another clustering on the physical-level features to form the estimations of the true labels on the physical layer. The clustering results on both layers can facilitate in tracking the changes in the uncertainties of the instances. Finally, the initially integrated labels that are likely to be wrongly inferred on the conceptual layer can be addressed using the estimated labels on the physical layer. The clustering processes on both layers can keep providing guidance information for each other in the multiple label remedy rounds. The experimental results on 12 real-world crowdsourcing data sets show that the performance of the proposed method in terms of accuracy is better than that of the state-of-the-art methods.

Multi-label active learning for image classification has been a popular research topic. It faces several challenges, even though related work has made great progress. Existing studies on multi-label active learning do not pay attention to the cleanness of sample data. In reality, data are easily polluted by external influences that are likely to disturb the exploration of data space and have a negative effect on model training. Previous methods of label correlation mining, which are purely based on observed label distribution, are defective. Apart from neglecting noise influence, they also cannot acquire sufficient relevant information. In fact, they neglect inner relation mapping from example space to label space, which is an implicit way of modeling label relationships. To solve these issues, we develop a novel multi-label active learning with low-rank application (ENMAL) algorithm in this paper. A low-rank model is constructed to quantize noise level, and the example-label pairs that contain less noise are emphasized when sampling. A low-rank mapping matrix is learned to signify the mapping relation of a multi-label domain to capture a more comprehensive and reasonable label correlation. Integrating label correlation with uncertainty and considering sample noise, an efficient sampling strategy is developed. We extend ENMAL with automatic labeling (denoted as AL-ENMAL) to further reduce the annotation workload of active learning. Empirical research demonstrates the efficacy of our approaches.

Distributed denial-of-service (DDoS) has caused major damage to cloud computing, and the false- and missing-alarm rates of existing DDoS attack-detection methods are relatively high in cloud environment. In this paper, we propose a DDoS attack-detection method with enhanced random forest (RF) optimized by genetic algorithm based on flow correlation degree (FCD) feature. We define the FCD feature according to the asymmetric and semidirectivity interaction characteristics and use the two-tuples FCD feature consisting of packet-statistical degree (PSD) and semidirectivity interaction abnormality (SDIA) to describe the features of attack flow and normal flow. Then we use a genetic algorithm based on the FCD feature sequences to optimize two key parameters of the decision tree in the RF: the maximum number of decision trees and the maximum depth of every single decision tree. We apply the trained RF model with optimized parameters to generate the classifier to be used for DDoS attack-detection. The experiment shows that the proposed method can effectively detect DDoS attacks in cloud environment with a higher accuracy rate and lower false- and missing-alarm rates compared to existing DDoS attack-detection methods.

Distributed denial of service (DDoS) attacks has caused huge economic losses to society. They have become one of the main threats to Internet security. Most of the current detection methods based on a single feature and fixed model parameters cannot effectively detect early DDoS attacks in cloud and big data environment. In this paper, an adaptive DDoS attack detection method (ADADM) based on multiple-kernel learning (MKL) is proposed. Based on the burstiness of DDoS attack flow, the distribution of addresses, and the interactivity of communication, we define five features to describe the network flow characteristic. Based on the ensemble learning framework, the weight of each dimension is adaptively adjusted by increasing the interclass mean with a gradient ascent and reducing the intraclass variance with a gradient descent, and the classifier is established to identify an early DDoS attack by training simple multiple-kernel learning (SMKL) models with two characteristics including interclass mean squared difference growth (M-SMKL) and intraclass variance descent (S-SMKL). The sliding window mechanism is used to coordinate the S-SMKL and M-SMKL to detect the early DDoS attack. The experimental results indicate that this method can detect DDoS attacks early and accurately.

Parameter in learning problems (usually arising from the tradeoff between training error minimization and regularization) is often tuned by cross validation (CV). A solution path provides a compact representation of all optimal solutions, which can be used to determine the parameter with the global minimum CV error, without solving original optimization problems multiple times based on grid search. However, existing solution path algorithms do not provide a unified implementation to various learning problems. In this paper, we first introduce a general parametric quadratic programming (PQP) problem that can be instantiated to an extensive number of learning problems. Then, we propose a generalized solution path (GSP) for the general PQP problem. Particularly, we use the $QR$ decomposition to handle singularities in GSP. Finally, we analyze the finite convergence and the time complexity of GSP. Our experimental results on a variety of data sets not only confirm the identicality between GSP and several existing solution path algorithms but also show the superiority of our GSP over the existing solution path algorithms on both generalization and robustness. Finally, we provide a practical guild of using the GSP to solve two important learning problems, i.e., generalized error path and Ivanov SVM.

Crowdsourcing systems provide a cost effective and convenient way to collect labels, but they often fail to guarantee the quality of the labels. This paper proposes a novel framework that introduces noise correction techniques to further improve the quality of integrated labels that are inferred from the multiple noisy labels of objects. In the proposed general framework, information about the qualities of labelers estimated by a front-end ground truth inference algorithm is utilized to supervise subsequent label noise filtering and correction. The framework uses a novel algorithm termed adaptive voting noise correction (AVNC) to precisely identify and correct the potential noisy labels. After filtering out the instances with noisy labels, the remaining cleansed data set is used to create multiple weak classifiers, based on which a powerful ensemble classifier is induced to correct these noises. Experimental results on eight simulated data sets with different kinds of features and two real-world crowdsourcing data sets in different domains consistently show that: 1) the proposed framework can improve label quality regardless of inference algorithms, especially under the circumstance that each instance has a few repeated labels and 2) since the proposed AVNC algorithm considers both the number of and the probability of potential label noises, it outperforms the state-of-the-art noise correction algorithms.

Existing studies show that extracting a complete keyphrase candidate set is the first and crucial step to extract high quality keyphrases from documents. Based on a common sense that words do not repeatedly appear in an effective keyphrase, we propose a novel algorithm named KCSP for document-specific keyphrase candidate search using sequential pattern mining with gap constraints, which only needs to scan a document once and automatically specifies appropriate gap constraints for words without users’ participation. The experimental results confirm that it helps improve the quality of keyphrase extraction.

With the rapid development of location-based social networks, Point-of-Interest (POI) recommendation has played an important role in helping people discover attractive locations. However, existing POI recommendation methods assume a flat structure of POIs, which are better described in a hierarchical structure in reality. Furthermore, we discover that both users' content and spatial preferences exhibit hierarchical structures. To this end, in this paper, we propose a hierarchical geographical matrix factorization model (HGMF) to utilize the hierarchical structures of both users and POIs for POI recommendation. Specifically, we first describe the POI influence degrees over regions with two-dimensional normal distribution, and learn the influence areas of different layers of POIs as the input of HGMF. Then, we perform matrix factorization on user content preference matrix, user spatial preference matrix, and POIs characteristic matrix jointly with the modeling of implicit hierarchical structures. Moreover, a two-step optimization method is proposed to learn the implicit hierarchical structure and find the solution of HGMF efficiently. Finally, we evaluate HGMF on two large-scale real-world location-based social networks datasets. Our experimental results demonstrate that it outperforms the state-of-the-art methods in terms of precision and recall.

In multi-label image classification, each image is always associated with multiple labels and labels are usually correlated with each other. The intrinsic relation among labels can definitely contribute to classifier training. However, most previous studies on active learning for multi-label image classification purely mine label correlation based on observed label distribution. They ignore the mapping relation between examples and their labels. This mapping relation also implicates label relationship. Ignoring the mapping relation leads to an uncomprehensive label correlation estimation and results in a bad performance for classification. In this paper, we propose a novel multi-label active learning with low-rank mapping for image classification, called LMMAL, to solve this issue. More precisely, we train a low-rank mapping matrix to signify the mapping relation between the feature space and the label space of a certain multi-label dataset. Using this low-rank mapping relation, we exploit a full label correlation. Subsequently, an effective sampling strategy is designed by integrating this potential information with uncertainty to select the most informative example-label pairs. In addition, we extend LMMAL with automatic labeling (denoted as AL-LMMAL) to further reduce the annotation workload of active learning. Empirical results demonstrate the effectiveness of our approaches.

Multi-label active learning for image classification has attracted great attention over recent years and a lot of relevant works are published continuously. However, there still remain some problems that need to be solved, such as existing multi-label active learning algorithms do not reflect on the cleanness of sample data and their ways on label correlation mining are defective. For one thing, sample data is usually contaminated in reality, which disturbs the estimation of data distribution and further hinders the model training. For another, previous approaches for label relationship exploration are purely based on the observed label distribution of an incomplete training set, which cannot provide sufficiently efficient information. To address these issues, we propose a novel adaptive low-rank multi-label active learning algorithm, called LRMAL. Specifically, we first use low-rank matrix recovery to learn an effective low-rank feature representation from the noisy data. In a subsequent sampling phase, we make use of its superiorities to evaluate the general informativeness of each unlabeled example-label pair. Based on an intrinsic mapping relation between the example space and the label space of a certain multi-label dataset, we recover the incomplete labels of a training set for a more comprehensive label correlation mining. Furthermore, to reduce the redundancy among the selected example-label pairs, we use a diversity measurement to diversify the sampled data. Finally, an effective sampling strategy is developed by integrating these two aspects of potential information with uncertainty based on an adaptive integration scheme. Experimental results demonstrate the effectiveness of our approach.

Recently accumulated massive amounts of geo-tagged photos provide an excellent opportunity to understand human behaviors and can be used for personalized tour recommendation. However, no existing work has considered the visual content information in these photos for tour recommendation. We believe the visual features of photos provide valuable information on measuring user / Point-of-Interest (POI) similarities, which is challenging due to data sparsity. To this end, in this paper, we propose a visual feature enhanced tour recommender system, named 'Photo2Trip', to utilize the visual contents and collaborative filtering models for recommendation. Specifically, we first extract various visual features from photos taken by tourists. Then, we propose a Visual-enhanced Probabilistic Matrix Factorization model (VPMF), which integrates visual features into the collaborative filtering model, to learn user interests by leveraging the historical travel records. Moreover, user interests together with trip constraints are formalized to an optimization problem for trip planning. Finally, the experimental results on real-world data show that our proposed visual-enhanced personalized tour recommendation method outperforms other benchmark methods in terms of recommendation accuracy. The results also show that visual features are effective on alleviating the data sparsity and cold start problems on personalized tour recommendation.

This paper proposes a novel general label aggregation method for both binary and multi-class labeling in crowdsourcing, namely Bi-Layer Clustering (BLC), which clusters two layers of features - the conceptual-level and the physical-level features - to infer true labels of instances. BLC first clusters the instances using the conceptual-level features extracted from their multiple noisy labels and then performs clustering again using the physical-level features. It can facilitate tracking the uncertainty changes of the instances, so that the integrated labels that are likely to be falsely inferred on the conceptual layer can be easily corrected using the estimated labels on the physical layer. Experimental results on two real-world crowdsourcing data sets show that BLC outperforms seven state-of-the-art methods.

Multi‐label image classification has attracted considerable attention in machine learning recently. Active learning is widely used in multi‐label learning because it can effectively reduce the human annotation workload required to construct high‐performance classifiers. However, annotation by experts is costly, especially when the number of labels in a dataset is large. Inspired by the idea of semi‐supervised learning, in this study, the authors propose a novel, semi‐supervised multi‐label active learning (SSMAL) method that combines automated annotation with human annotation to reduce the annotation workload associated with the active learning process. In SSMAL, they capture three aspects of potentially useful information – classification prediction information, label correlation information, and example spatial information – and they use this information to develop an effective strategy for automated annotation of selected unlabelled example‐label pairs. The experimental results obtained in this study demonstrate the effectiveness of the authors' proposed approach.

Model selection plays an important role in cost-sensitive SVM (CS-SVM). It has been proven that the global minimum cross validation (CV) error can be efficiently computed based on the solution path for one parameter learning problems. However, it is a challenge to obtain the global minimum CV error for CS-SVM based on one-dimensional solution path and traditional grid search, because CS-SVM is with two regularization parameters. In this paper, we propose a solution and error surfaces based CV approach (CV-SES). More specifically, we first compute a two-dimensional solution surface for CS-SVM based on a bi-parameter space partition algorithm, which can fit solutions of CS-SVM for all values of both regularization parameters. Then, we compute a two-dimensional validation error surface for each CV fold, which can fit validation errors of CS-SVM for all values of both regularization parameters. Finally, we obtain the CV error surface by superposing K validation error surfaces, which can find the global minimum CV error of CS-SVM. Experiments are conducted on seven datasets for cost sensitive learning and on four datasets for imbalanced learning. Experimental results not only show that our proposed CV-SES has a better generalization ability than CS-SVM with various hybrids between grid search and solution path methods, and than recent proposed cost-sensitive hinge loss SVM with three-dimensional grid search, but also show that CV-SES uses less running time.

Multilabel image classification has been a hot topic in the field of computer vision and image understanding in recent years. To achieve better classification performance with fewer labeled images, multilabel active learning is used for this scenario. Several active learning methods have been proposed for multilabel image classification. However, all of them assume that either all training images have complete labels or label correlations are given at the beginning. These two assumptions are unrealistic. In fact, it is very difficult to obtain complete labels for each example, in particular when the size of labels in a multilabel dataset is very large. Typically, only partial labels are available. This is one type of “weak label” problem. To solve this weak label problem inside multilabel active learning, this paper proposes a novel solution called AE-WLMAL. AE-WLMAL explores conditional label correlations on the weak label problem with the help of input features and then utilizes label correlations to construct a unified sampling strategy and evaluate the informativeness of each example-label pair in a multilabel dataset for active sampling. In addition, a pruning strategy is adopted to further improve its computation efficiency. Moreover, AE-WLAML exploits label correlations to infer labels for unlabeled images, which further reduces human labeling cost. Our experimental results on seven real-world datasets show that AE-WLMAL consistently outperforms existing approaches.

The ν -support vector classification has the advantage of using a regularization parameter ν to control the number of support vectors and margin errors. Recently, a regularization path algorithm for ν -support vector classification ( ν -SvcPath) suffers exceptions and singularities in some special cases. In this brief, we first present a new equivalent dual formulation for ν -SVC and, then, propose a robust ν -SvcPath, based on lower upper decomposition with partial pivoting. Theoretical analysis and experimental results verify that our proposed robust regularization path algorithm can avoid the exceptions completely, handle the singularities in the key matrix, and fit the entire solution path in a finite number of steps. Experimental results also show that our proposed algorithm fits the entire solution path with fewer steps and less running time than original one does.

Minimax probability machine (MPM) is an interesting discriminative classifier based on generative prior knowledge. It can directly estimate the probabilistic accuracy bound by minimizing the maximum probability of misclassification. The structural information of data is an effective way to represent prior knowledge, and has been found to be vital for designing classifiers in real-world problems. However, MPM only considers the prior probability distribution of each class with a given mean and covariance matrix, which does not efficiently exploit the structural information of data. In this paper, we use two finite mixture models to capture the structural information of the data from binary classification. For each subdistribution in a finite mixture model, only its mean and covariance matrix are assumed to be known. Based on the finite mixture models, we propose a structural MPM (SMPM). SMPM can be solved effectively by a sequence of the second-order cone programming problems. Moreover, we extend a linear model of SMPM to a nonlinear model by exploiting kernelization techniques. We also show that the SMPM can be interpreted as a large margin classifier and can be transformed to support vector machine and maxi-min margin machine under certain special conditions. Experimental results on both synthetic and real-world data sets demonstrate the effectiveness of SMPM.

Image classification is a hot topic of pattern recognition in computer vision. In order to achieve high accuracy of classification, a certain amount of high quality pictures are needed. As a matter of fact, high quality pictures are scarce. Active learning can solve such a problem. Label dependences play an important role in multi-label active learning for image classification. The interdependences between different labels are usually different and asymmetrical. This paper first brings the asymmetrical conditional label dependences into a novel active learning method for multi-label image classification based on the asymmetrical conditional label dependence, called ACDAL. Our extensive experimental results on three image and two non-image datasets show that our new approach ACDAL significantly outperforms existing approaches.

Multi-label classification is to assign an instance to multiple classes. Naive Bayes (NB) is one of the most popular algorithms for pattern recognition and classification. It has a high performance in single label classification. It is naturally extended for multi-label classification under the assumption of label independence. As we know, NB is based on a simple but unrealistic assumption that attributes are conditionally independent given the class. Therefore, a double weighted NB (DWNB) is proposed to demonstrate the influences of predicting different labels based on different attributes. Our DWNB utilizes the niching cultural algorithm (NLA) to determine the weight configuration automatically. Our experimental results show that our proposed DWNB outperforms NB and its extensions significantly in multi-label classification.

Due to low quality of crowdsourced labelers, the integrated label of each example is usually inferred from its multiple noisy labels provided by different labelers. This paper proposes a novel algorithm, Ground Truth Inference using Clustering (GTIC), to improve the quality of integrated labels for multi-class labeling. For a K labeling case, GTIC utilizes the multiple noisy label sets of examples to generate features. Then, it uses a K-Means algorithm to cluster all examples into K different groups, each of which is mapped to a specific class. Examples in the same cluster are assigned a corresponding class label. We compare GTIC with four existing multi-class ground truth inference algorithms, majority voting (MV), Dawid & Skene's (DS), ZenCrowd (ZC) and Spectral DS (SDS), on one synthetic and eight real-world datasets. Experimental results show that the performance of GTIC is significantly superior to the others in terms of both accuracy and M-AUC. Besides, the running time of GTIC is about twenty times faster than EM-based complicated inference algorithms.

Learning from imbalanced multilabel data is a challenging task. It has attracted considerable attention recently. In this paper we propose a MultiLabel Best First Over-sampling (ML-BFO) to improve the performance of multilabel classification algorithms, based on imbalance minimization and Wilson's ENN rule. Our experimental results show that ML-BFO not only duplicates fewer samples but also reduces the imbalance level much more than two state-of-the-art multilabel sampling methods, i.e., an over-sampling method LP-ROS and an under-sampling method MLeNN. Besides, ML-BFO significantly improves the performance of multilabel classification algorithms, and performs much better than LP-ROS and MLeNN.

This paper proposes a novel framework that introduces noise correction techniques to further improve label quality after ground truth inference in crowdsourcing. In the framework, an adaptive voting noise correction algorithm (AVNC) is proposed to identify and correct the most likely noises with the help of estimated qualities of labelers provided by the ground truth inference. The experimental results on two real-world datasets show that (1) the framework can improve label quality regardless of inference algorithms, especially under the circumstance that each example has a few noisy labels; and (2) since the algorithm AVNC considers both the number of and the probability of potential noises, it outperforms a baseline noise correction algorithm.

The important task of correcting label noise is addressed infrequently in literature. The difficulty of developing a robust label correction algorithm leads to this silence concerning label correction. To break the silence, we propose two algorithms to correct label noise. One utilizes self-training to re-label noise, called Self-Training Correction (STC). Another is a clustering-based method, which groups instances together to infer their ground-truth labels, called Cluster-based Correction (CC). We also adapt an algorithm from previous work, a consensus-based method called Polishing that consults with an ensemble of classifiers to change the values of attributes and labels. We simplify Polishing such that it only alters labels of instances, and call it Polishing Labels (PL). We experimentally compare our novel methods with Polishing Labels by examining their improvements on the label qualities, model qualities, and AUC metrics of binary and multi-class data sets, and ultimately conclude that only CC can significantly improve label qualities, model qualities, and AUC metrics consistently. STC and PL can improve these metrics in some cases, but not as reliably. Hence, our Cluster-based Correction method is the best.

We investigate the methods of improving data quality, in terms of label accuracy, in the context of image labeling in crowdsourcing. First, we look at three consensus methods for inferring a ground-truth label from the multiple noisy labels obtained from crowdsourcing, i.e., Majority Voting (MV), Dawid Skene (DS), and KOS. We then apply three noise correction methods to correct labels inferred by these consensus methods, i.e., Polishing Labels (PL), Self-Training Correction (STC), and Cluster Correction (CC). Our experimental results show that the noise correction methods improve the labeling quality significantly.

Label correlation analysis is very important for multi-label classification. And there is no study to measure the label correlation for example-label based active learning. In this paper, from a statistical point of view, we proposed a cosine similarity based multi-label active learning (CosMAL), which uses cosine similarity to accurately evaluate the correlations between all labels. It further uses the average correlation between the potential label and the other unlabeled labels as the label information for each sample-label pair. And then we select the most informativeness example-label pairs. Our empirical results demonstrate that our proposed method CosMAL outperforms the state-of-the-art active learning for multi-label classification. It significantly reduces the labeling workload and improves the performance of a classifier learned.

In general, the first step for supervised learning from crowdsourced data is integration. To obtain training data as traditional machine learning, the ground truth for each example in the crowdsourcing dataset must be integrated with consensus algorithms. However, some information and correlations among labels in the crowdsourcing dataset have discarded after integration. In order to study whether the information and correlations are useful for learning, we proposed three types of neural networks. Experimental results show that i) all the three types of neural networks have abilities to predict labels for future unseen examples, ii) when labelers have lower qualities, the information and correlations in crowdsourcing datasets, which are discarded by integration, does improve the performance of neural networks significantly, iii) when labelers have higher label qualities, the information and correlations have little impact on improving accuracy of neural networks.

With various online crowdsourcing platforms, it is easy to collect multiple labels for the same examples from the crowd. Consensus integration algorithms can infer the estimated ground truths from the multiple label sets of these crowdsourcing datasets. However, it couldn't be avoided that these integrated (estimated) labels still contain noises. In order to further improve the performance of a model learned from data with these integrated labels, we propose an active learning framework to further improve the data quality, such that to improve the model quality, through acquiring limited true labels from experts (the oracle). We further investigate two active learning strategies in terms of two uncertainty measures (i.e., CLUE and MUE) within the active learning framework. From our experimental results on eight simulation crowdsourcing datasets and four real-world crowdsourcing datasets with three popular consensus integration algorithms, we draw several conclusions as follows. (i) Our active learning framework with the input from the oracle significantly improves the generalization ability of the model learned from crowdsourcing data. (ii) Our two active learning strategies outperform a random active learning strategy.

Active learning is to select the most informative examples to request their labels. Most previous studies in active learning for multi-label classification didn't pay enough attention on label correlations. This leads to a bad performance for classification. In this paper, we proposed a chi-square statistics multi-label active learning (CSMAL) algorithm, which uses chi-square statistics to accurately evaluate correlations between labels. CSMAL considers not only positive relationships but also negative ones. It uses the average correlation between a potential label and its rest unlabeled labels as the label information for each sample-label pair. CSMAL further integrates uncertainty and label information to select example-label pairs to request labels. Our empirical results demonstrate that our proposed method CSMAL outperforms the state-of-the-art active learning methods for multi-label classification. It significantly reduces the labeling workloads and improves the performance of a classifier built.

In order to achieve better classification performance with even fewer labeled images, active learning is suitable for these situations. Several active learning methods have been proposed for multi-label image classification, but all of them assume that all training images with complete labels. However, as a matter of fact, it is very difficult to get complete labels for each example, especially when the size of labels in a multi-label domain is huge. Usually, only partial labels are available. This is one kind of "weak label" problems. This paper proposes an ingeniously solution to this "weak label" problem on multi-label active learning for image classification (called WLMAL). It explores label correlation on the weak label problem with the help of input features, and then utilizes label correlation to evaluate the informativeness of each example-label pair in a multi-label dataset for active sampling. Our experimental results on three real-world datasets show that our proposed approach WLMAL consistently outperforms existing approaches significantly.

With crowdsourcing systems, it is easy to collect multiple noisy labels for the same object for supervised learning. This dynamic annotation procedure fits the active learning perspective and accompanies the imbalanced multiple noisy labeling problem. This paper proposes a novel active learning framework with multiple imperfect annotators involved in crowdsourcing systems. The framework contains two core procedures: label integration and instance selection. In the label integration procedure, a positive label threshold (PLAT) algorithm is introduced to induce the class membership from the multiple noisy label set of each instance in a training set. PLAT solves the imbalanced labeling problem by dynamically adjusting the threshold for determining the class membership of an example. Furthermore, three novel instance selection strategies are proposed to adapt PLAT for improving the learning performance. These strategies are respectively based on the uncertainty derived from the multiple labels, the uncertainty derived from the learned model, and the combination method (CFI). Experimental results on 12 datasets with different underlying class distributions demonstrate that the three novel instance selection strategies significantly improve the learning performance, and CFI has the best performance when labeling behaviors exhibit different levels of imbalance in crowdsourcing systems. We also apply our methods to a real-world scenario, obtaining noisy labels from Amazon Mechanical Turk, and show that our proposed strategies achieve very high performance.

It can be easy to collect multiple noisy labels for the same object via Internet-based crowdsourcing systems. Labelers may have bias when labeling, due to lacking expertise, dedication, and personal preference. These cause Imbalanced Multiple Noisy Labeling. In most cases, we have no information about the labeling qualities of labelers and the underlying class distributions. It is important to design agnostic solutions to utilize these noisy labels for supervised learning. We first investigate how imbalanced multiple noisy labeling affects the class distributions of training sets and the performance of classification. Then, an agnostic algorithm Positive LAbel frequency Threshold (PLAT) is proposed to deal with the imbalanced labeling issue. Simulations on eight UCI data sets with different underlying class distributions show that PLAT not only effectively deals with the imbalanced multiple noisy labeling problems that off-the-shelf agnostic methods cannot cope with, but also performs nearly the same as majority voting under the circumstances without imbalance. We also apply PLAT to eight real-world data sets with imbalanced labels collected from Amazon Mechanical Turk, and the experimental results show that PLAT is efficient and better than other ground truth inference algorithms.

Support vector ordinal regression (SVOR) is a popular method to tackle ordinal regression problems. However, until now there were no effective algorithms proposed to address incremental SVOR learning due to the complicated formulations of SVOR. Recently, an interesting accurate on-line algorithm was proposed for training ν -support vector classification (ν-SVC), which can handle a quadratic formulation with a pair of equality constraints. In this paper, we first present a modified SVOR formulation based on a sum-of-margins strategy. The formulation has multiple constraints, and each constraint includes a mixture of an equality and an inequality. Then, we extend the accurate on-line ν-SVC algorithm to the modified formulation, and propose an effective incremental SVOR algorithm. The algorithm can handle a quadratic formulation with multiple constraints, where each constraint is constituted of an equality and an inequality. More importantly, it tackles the conflicts between the equality and inequality constraints. We also provide the finite convergence analysis for the algorithm. Numerical experiments on the several benchmark and real-world data sets show that the incremental algorithm can converge to the optimal solution in a finite number of steps, and is faster than the existing batch and incremental SVOR algorithms. Meanwhile, the modified formulation has better accuracy than the existing incremental SVOR algorithm, and is as accurate as the sum-of-margins based formulation of Shashua and Levin.

Multi-label image data is becoming ubiquitous. Image semantic understanding is typically formulated as a classification problem. This paper focuses on multi-label active learning for image classification. It first extends a traditional example based active learning method for multilabel active learning for image classification. Since the traditional example based active method doesn't work well, we propose a novel example-label based multi-label active learning method. Our experimental results on two image datasets demonstrate that the proposed method significantly reduces the labeling workload and improves the performance of the built classifier. Additionally, we conduct experiments on two other types of multi-label datasets for validating the versatility of our proposed method, and the experimental results show the consistent effect.

One-against-all and one-against-one are two popular methodologies for reducing multiclass classification problems into a set of binary classifications. In this paper, we are interested in the performance of both one-against-all and one-against-one for classification algorithms, such as decision tree, naïve bayes, support vector machine, and logistic regression. Since both one-against-all and one-against-one work like creating a classification committee, they are expected to improve the performance of classification algorithms. However, our experimental results surprisingly show that one-against-all worsens the performance of the algorithms on most datasets. One-against-one helps, but performs worse than the same iterations of bagging these algorithms. Thus, we conclude that both one-against-all and one-against-one should not be used for the algorithms that can perform multiclass classifications directly. Bagging is better approach for improving their performance.

Multi-label classifications exist in many real world applications. This paper empirically studies the performance of a variety of multi-label classification algorithms. Some of them are developed based on problem transformation. Some of them are developed based on adaption. Our experimental results show that the adaptive Multi-Label K-Nearest Neighbor performs the best, followed by Random k-Label Set, followed by Classifier Chain and Binary Relevance. Adaboost.MH performs the worst, followed by Pruned Problem Transformation. Our experimental results also provide us the confidence of the correlations among multi-labels. These insights shed light for future research directions on multi-label classifications.

When labeling objects via Internet-based outsourcing systems, the labelers may have bias, because they lack expertise, dedication and personal preference. These reasons cause Imbalanced Multiple Noisy Labeling. To deal with the imbalance labeling issue, we propose an agnostic algorithm PLAT (Positive LAbel frequency Threshold) which does not need any information about quality of labelers and underlying class distribution. Simulations on eight real-world datasets with different underlying class distributions demonstrate that PLAT not only effectively deals with the imbalanced multiple noisy labeling problem that off-the-shelf agnostic methods cannot cope with, but also performs nearly the same as majority voting under the circumstances that labelers have no bias.

Internet-based crowdsourcing systems can be viewed as a kind of loosely coupled social networks. With these systems, it is easy to collect multiple noisy labels for the same object when conducting annotation for supervised learning. Because non-expert labelers lack expertise and dedication, and have strong personal preference, they may have bias when labeling. These cause Imbalanced Multiple Noisy Labeling. In this paper, we propose an agnostic algorithm Positive LAbel frequency Threshold (PLAT) to deal with imbalanced labeling. Because of the dynamics of social networks, in most cases no information about the qualities of labelers and underlying class distributions can be acquired. PLAT does not require prior knowledge of the labeling qualities of labelers, the underlying class distributions, and the level of labeling imbalance. Simulations on eight real-world datasets with different underlying class distributions demonstrate that PLAT not only effectively deals with the imbalanced multiple noisy labeling that off-the-shelf agnostic methods cannot cope with, but also performs nearly the same as majority voting under the circumstances that labelers have no bias.

The ν-support vector machine ( ν-SVM) for classification has the advantage of using a parameter ν on controlling the number of support vectors and margin errors. Recently, an interesting accurate on-line algorithm accurate on-line ν-SVM algorithm (AONSVM) is proposed for training ν-SVM. AONSVM can be viewed as a special case of parametric quadratic programming techniques. It is demonstrated that AONSVM avoids the infeasible updating path as far as possible, and successfully converges to the optimal solution based on experimental analysis. However, because of the differences between AONSVM and classical parametric quadratic programming techniques, there is no theoretical justification for these conclusions. In this paper, we prove the feasibility and finite convergence of AONSVM under two assumptions. The main results of feasibility analysis include: 1) the inverses of the two key matrices in AONSVM always exist; 2) the rules for updating the two key inverse matrices are reliable; 3) the variable ζ can control the adjustment of the sum of all the weights efficiently; and 4) a sample cannot migrate back and forth in successive adjustment steps among the set of margin support vectors, the set of error support vectors, and the set of the remaining vectors. Moreover, the analyses of AONSVM also provide the proofs of the feasibility and finite convergence for accurate on-line C-SVM learning directly.

The K-Means algorithms are fundamental in machine learning and data mining. In this study, we investigate interval-valued rather than commonly used point-valued centroids in the K-Means algorithm. Using a proposed interval peak method to select initial interval centroids, we have obtained overall quality improvement of clusters on a set of test problems in the Fundamental Clustering Problem Suite (FCPS).

With the outsourcing of small tasks becoming easier, it is possible to obtain non-expert/imperfect labels at low cost. With low-cost imperfect labeling, it is straightforward to collect multiple labels for the same data items. This paper addresses the strategies of utilizing these multiple labels for improving the performance of supervised learning, based on two basic ideas: majority voting and pair wise solutions. We show several interesting results based on our experiments. The soft majority voting strategies can reduce the bias and roughness, and improve the performance of the directed hard majority voting strategy. Pair wise strategies can completely avoid the bias by having both sides (potential correct and incorrect/noisy information) considered (for binary classification). They have very good performance whenever there are a few or many labels available. However, it could also keep the noise. The improved variation that reduces the impact of the noisy information is recommended. All five strategies investigated are labeling quality agnostic strategies, and can be applied to real world applications directly. The experimental results show some of them perform better than or at least very close to the gnostic strategies.

This paper addresses the repeated acquisition of labels for data items when the labeling is imperfect. We examine the improvement (or lack thereof) in data quality via repeated labeling, and focus especially on the improvement of training labels for supervised induction. With the outsourcing of small tasks becoming easier, for example via Rent-A-Coder or Amazon's Mechanical Turk, it often is possible to obtain less-than-expert labeling at low cost. With low-cost labeling, preparing the unlabeled part of the data can become considerably more expensive than labeling. We present repeated-labeling strategies of increasing complexity, and show several main results. (i) Repeated-labeling can improve label quality and model quality, but not always. (ii) When labels are noisy, repeated labeling can be preferable to single labeling even in the traditional setting where labels are not particularly cheap. (iii) As soon as the cost of processing the unlabeled data is not free, even the simple strategy of labeling everything multiple times can give considerable advantage. (iv) Repeatedly labeling a carefully chosen set of points is generally preferable, and we present a robust technique that combines different notions of uncertainty to select data points for which quality should be improved. The bottom line: the results show clearly that when labeling is not perfect, selective acquisition of multiple labels is a strategy that data miners should have in their repertoire; for certain label-quality/cost regimes, the benefit is substantial.

It is often expensive to acquire data in real-world data mining applications. Most previous data mining and machine learning research, however, assumes that a fixed set of training examples is given. In this paper, we propose an online cost-sensitive framework that allows a learner to dynamically acquire examples as it learns, and to decide the ideal number of examples needed to minimize the total cost. We also propose a new strategy for Partial Example Acquisition (PAS), in which the learner can acquire examples with a subset of attribute values to reduce the data acquisition cost. Experiments on UCI datasets show that the new PAS strategy is an effective method in reducing the total cost for data acquisition.

In medical diagnosis, doctors often have to order sets of medical tests in sequence in order to make an accurate diagnosis of patient diseases. While doing so they have to make a trade-off between the cost of the tests and possible misdiagnosis. In this paper, we use cost-sensitive learning to model this process. We assume that test examples (new patients) may contain missing values, and their actual values can be acquired at cost (similar to doing medical tests) in order to reduce misclassification errors (misdiagnosis). We propose a novel Sequential Batch Test algorithm that can acquire sets of attribute values in sequence, similar to sets of medical tests ordered by doctors in sequence. The goal of our algorithm is to minimize the total cost (i.e., the trade-off) of acquiring attribute values and misclassifications. We demonstrate the effectiveness of our algorithm, and show that it outperforms previous methods significantly. Our algorithm can be readily applied in real-world diagnosis tasks. A case study on the heart disease is given in the paper.

While most software defects (i.e., bugs) are corrected and tested as part of the lengthy software development cycle, enterprise software vendors often have to release software products before all reported defects are corrected, due to deadlines and limited resources. A small number of these defects will be escalated by customers and they must be resolved immediately by the software vendors at a very high cost. In this paper, we develop an Escalation Prediction (EP) system that mines historic defect report data and predict the escalation risk of the defects for maximum net profit. More specifically, we first describe a simple and general framework to convert the maximum net profit problem to cost-sensitive learning. We then apply and compare several well-known cost-sensitive learning approaches for EP. Our experiments suggest that the cost-sensitive decision tree is the best method for producing the highest positive net profit and comprehensible results. The EP system has been deployed successfully in the product group of an enterprise software vendor.

In medical diagnosis, doctors must often determine what medical tests (e.g., X-ray and blood tests) should be ordered for a patient to minimize the total cost of medical tests and misdiagnosis. In this paper, we design cost-sensitive machine learning algorithms to model this learning and diagnosis process. Medical tests are like attributes in machine learning whose values may be obtained at a cost (attribute cost), and misdiagnoses are like misclassifications which may also incur a cost (misclassification cost). We first propose a lazy decision tree learning algorithm that minimizes the sum of attribute costs and misclassification costs. Then, we design several novel "test strategies" that can request to obtain values of unknown attributes at a cost (similar to doctors' ordering of medical tests at a cost) in order to minimize the total cost for test examples (new patients). These test strategies correspond to different situations in real-world diagnoses. We empirically evaluate these test strategies, and show that they are effective and outperform previous methods. Our results can be readily applied to real-world diagnosis tasks. A case study on heart disease is given throughout the paper