Composed image retrieval (CoIR) involves a multi-modal query of the reference image and modification text describing the desired changes, allowing users to express image retrieval intents flexibly and effectively. The key of CoIR lies in how to properly reason the search intent from the multi-modal query. Existing work either aligns the composite embedding of the multi-modal query and the target image embedding in the visual domain through late-fusion or converts all images into text descriptions and leverage large language models (LLM) for text semantic reasoning. However, this single-modality reasoning approach fails to comprehensively and interpretably capture the users’ ambiguous and uncertain intents in the multi-modal queries, incurring the inconsistency between retrieved results and ground truth. Besides, the expensive manually annotated datasets limit the further performance improvement of CoIR. To this end, this article proposes an LLM-enhanced Intent Uncertainty-Aware Linguistic-Visual Dual Channel Matching Model (IUDC), which combines the strengths of multi-modal late-fusion and LLMs for CoIR. We first construct an LLM-based triplet augmentation strategy to generate more synthetic training triplets. Based on this, the core of IUDC consists of two matching channels: the semantic matching channel is responsible for intent reasoning on the aspect-level attributes extracted by an LLM, and the visual matching channel accounts for the fine-grained visual matching between multi-modal fusion embedding and target images. Considering the intent uncertainty presented in the multi-modal queries, we introduce Probability Distribution Encoder (PDE) to project the intents as probabilistic distributions in the two matching channels. Consequently, a mutually enhanced module is designed to share knowledge between the visual and semantic representations for better representation learning. Finally, the matching scores of two channels are added to retrieve the target image. Extensive experiments conducted on two real datasets demonstrate the effectiveness and superiority of our model. Notably, with the help of the proposed LLM-based triplet augmentation strategy, our model achieves a new record of state-of-the-art performance among all datasets.

Music recommender systems play a critical role in music streaming platforms by providing users with music that they are likely to enjoy. Recent studies have shown that user emotions can influence users’ preferences for music moods. However, existing emotion-aware music recommender systems (EMRSs) explicitly or implicitly assume that users’ actual emotional states expressed through identical emotional words are homogeneous. They also assume that users’ music mood preferences are homogeneous under the same emotional state. In this article, we propose four types of heterogeneity that an EMRS should account for: emotion heterogeneity across users, emotion heterogeneity within a user, music mood preference heterogeneity across users, and music mood preference heterogeneity within a user. We further propose a Heterogeneity-aware Deep Bayesian Network (HDBN) to model these assumptions. The HDBN mimics a user’s decision process of choosing music with four components: personalized prior user emotion distribution modeling, posterior user emotion distribution modeling, user grouping, and Bayesian neural network-based music mood preference prediction. We constructed two datasets, called EmoMusicLJ and EmoMusicLJ-small, to validate our method. Extensive experiments demonstrate that our method significantly outperforms baseline approaches on metrics of HR, Precision, NDCG, and MRR. Ablation studies and case studies further validate the effectiveness of our HDBN. The source code and datasets are available at https://github.com/jingrk/HDBN .

With the wide use of Location-Based Social Networks (LBSNs), predicting user friendship from online social relations and offline trajectory data is of great value to improve the platform service quality and user satisfaction. Existing methods mainly focus on some hand-crafted features or graph embedding models based on the user-location bipartite graph, which cannot precisely capture the latent mobility similarity for the majority of users who have no explicit co-visit behaviors and also fail to balance the tradeoff between social features and mobility features for friendship prediction. In this regard, we propose a dual subgraph-based pairwise graph neural network (DSGNN) for friendship prediction in LBSNs, which extracts a pairwise social subgraph and a trajectory subgraph to model the social proximity and mobility similarity, respectively. Specifically, to overcome the co-visit data sparsity, we design an entropy-based random walk to construct a location graph that captures the high-level correlation between locations. Based on this, we characterize the pairwise mobility similarity from trajectory level instead of location level, which is modeled by a graph neural network (GNN) on a labeled trajectory subgraph composed of the two trajectories of the target user pair. Besides, we also utilize another GNN to extract social proximity based on social subgraph of the target user pair. Finally, we propose a gate layer to adaptively balance the fusion of the social and mobility features for friendship prediction. We conduct extensive experiments on the real-world datasets and demonstrate the superiority of our approach, which outperforms other state-of-the-art methods. In particular, the comparative experiments on the trajectory level mobility similarity further validate the effectiveness of the designed trajectory subgraph-based method, which can extract predictive mobility features.

Deep learning (DL) models have significantly improved the performance of text classification and text regression tasks. However, DL models are often strikingly vulnerable to adversarial attacks. Many researchers have aimed to develop adversarial attacks against DL models in realistic black-box settings (i.e., assuming no model knowledge is accessible to attackers). These attacks typically operate with a two-phase framework: (1) sensitivity estimation through gradient-based or deletion-based methods to evaluate the sensitivity of each token to the prediction of the target model, and (2) perturbation execution to craft adversarial examples based on the estimated token sensitivity. However, gradient-based and deletion-based methods used to estimate sensitivity often face issues of capturing token directionality and overlapping token sensitivities, respectively. In this study, we propose a novel eXplanation-based method for Adversarial Text Attacks (XATA) that leverages additive feature attribution explainable methods, namely LIME or SHAP, to measure the sensitivity of input tokens when crafting black-box adversarial attacks on DL models performing text classification or text regression. We evaluated XATA's attack performance on DL models executing text classification on the IMDB Movie Review, Yelp Reviews-Polarity, and Amazon Reviews-Polarity datasets and DL models conducting text regression on the My Personality, Drug Review, and CommonLit Readability datasets. The proposed XATA outperformed the existing gradient-based and deletion-based adversarial attack baselines in both tasks. These findings indicate that the ever-growing research focused on improving the explainability of DL models with additive feature attribution explainable methods can provide attackers with weapons to launch targeted adversarial attacks.

Online social media provides rich and varied information reflecting the significant concerns of the public during the coronavirus pandemic. Analyzing what the public is concerned with from social media information can support policy-makers to maintain the stability of the social economy and life of the society. In this article, we focus on the detection of the network public opinions during the coronavirus pandemic. We propose a novel Relational Topic Model for Short texts (RTMS) to draw opinion topics from social media data. RTMS exploits the feature of texts in online social media and the opinion propagation patterns among individuals. Moreover, a dynamic version of RTMS (DRTMS) is proposed to capture the evolution of public opinions. Our experiment is conducted on a real-world dataset which includes 67,592 comments from 14,992 users. The results demonstrate that, compared with the benchmark methods, the proposed RTMS and DRTMS models can detect meaningful public opinions by leveraging the feature of social media data. It can also effectively capture the evolution of public concerns during different phases of the coronavirus pandemic.

Outlier detection is an important task in data mining, and many technologies for it have been explored in various applications. However, owing to the default assumption that outliers are not concentrated, unsupervised outlier detection may not correctly identify group anomalies with higher levels of density. Although high detection rates and optimal parameters can usually be achieved by using supervised outlier detection, obtaining a sufficient number of correct labels is a time-consuming task. To solve these problems, we focus on semi-supervised outlier detection with few identified anomalies and a large amount of unlabeled data. The task of semi-supervised outlier detection is first decomposed into the detection of discrete anomalies and that of partially identified group anomalies, and a distribution construction sub-module and a data augmentation sub-module are then proposed to identify them, respectively. In this way, the dual multiple generative adversarial networks (Dual-MGAN) that combine the two sub-modules can identify discrete as well as partially identified group anomalies. In addition, in view of the difficulty of determining the stop node of training, two evaluation indicators are introduced to evaluate the training status of the sub-GANs. Extensive experiments on synthetic and real-world data show that the proposed Dual-MGAN can significantly improve the accuracy of outlier detection, and the proposed evaluation indicators can reflect the training status of the sub-GANs.

Personalized recommendation has become more and more important for users to quickly find relevant items. The key issue of the recommender system is how to model user preferences. Previous work mostly employed user historical data to learn users’ preferences, but faced with the data sparsity problem. The prevalence of online social networks promotes increasing online discussion groups, and users in the same group often have similar interests and preferences. Therefore, it is necessary to integrate group information for personalized recommendation. The existing work on group-information-enhanced recommender systems mainly relies on the item information related to the group, which is not expressive enough to capture the complicated preference dependency relationships between group users and the target user. In this article, we solve the problem with the graph neural networks. Specifically, the relationship between users and items, the item preferences of groups, and the groups that users participate in are constructed as bipartite graphs, respectively, and the user preferences for items are learned end to end through the graph neural network. The experimental results on the Last.fm and Douban Movie datasets show that considering group preferences can improve the recommendation performance and demonstrate the superiority on sparse users compared

Abstract With the development of Web‐based social networks, many personalized tag recommendation approaches based on multi‐information have been proposed. Due to the differences in users' preferences, different users care about different kinds of information. In the meantime, different elements within each kind of information are differentially informative for user tagging behaviors. In this context, how to effectively integrate different elements and different information separately becomes a key part of tag recommendation. However, the existing methods ignore this key part. In order to address this problem, we propose a deep neural network for tag recommendation. Specifically, we model two important attentive aspects with a hierarchical attention model. For different user‐item pairs, the bottom layered attention network models the influence of different elements on the features representation of the information while the top layered attention network models the attentive scores of different information. To verify the effectiveness of the proposed method, we conduct extensive experiments on two real‐world data sets. The results show that using attention network and different kinds of information can significantly improve the performance of the recommendation model, and verify the effectiveness and superiority of our proposed model.

Outlier detection is an important topic in machine learning and has been used in a wide range of applications. In this paper, we approach outlier detection as a binary-classification issue by sampling potential outliers from a uniform reference distribution. However, due to the sparsity of data in high-dimensional space, a limited number of potential outliers may fail to provide sufficient information to assist the classifier in describing a boundary that can separate outliers from normal data effectively. To address this, we propose a novel Single-Objective Generative Adversarial Active Learning (SO-GAAL) method for outlier detection, which can directly generate informative potential outliers based on the mini-max game between a generator and a discriminator. Moreover, to prevent the generator from falling into the mode collapsing problem, the stop node of training should be determined when SO-GAAL is able to provide sufficient information. But without any prior information, it is extremely difficult for SO-GAAL. Therefore, we expand the network structure of SO-GAAL from a single generator to multiple generators with different objectives (MO-GAAL), which can generate a reasonable reference distribution for the whole dataset. We empirically compare the proposed approach with several state-of-the-art outlier detection methods on both synthetic and real-world datasets. The results show that MO-GAAL outperforms its competitors in the majority of cases, especially for datasets with various cluster types or high irrelevant variable ratio. The experiment codes are available at: https://github.com/leibinghe/GAAL-based-outlier-detection.

Role analysis plays an important role when exploring social media and knowledge‐sharing platforms for designing marking strategies. However, current methods in role analysis have overlooked content generated by users (e.g., posts) in social media and hence focus more on user behavior analysis. The user‐generated content is very important for characterizing users. In this paper, we propose a novel method which integrates both user behavior and posted content by users to identify roles in online social networks. The proposed method models a role as a joint distribution of Gaussian distribution and multinomial distribution, which represent user behavioral feature and content feature respectively. The proposed method can be used to determine the number of roles concerned automatically. The experimental results show that the proposed method can be used to identify various roles more effectively and to get more insights on such characteristics.