Existing Text-Video Retrieval (TVR) methods predominantly rely on global frame representations, often disregarding the fine-grained temporal variations required for precise patch-level alignment. This is critical as video motion is inherently spatially localized; consequently, coarse frame-level modeling tends to be dominated by static backgrounds, overshadowing salient action cues. To address this limitation, we propose TRFG, a novel framework for text-video retrieval that addresses the challenges of modeling Temporal Reasoning and Fine-Grained cross-modal alignment. First, our Temporal Difference module captures frame-to-frame variations at the patch level, effectively suppressing static background noise to highlight "active" motion regions. Second, these differential signals are synthesized via a Temporal Aggregation module to form a coherent representation of the event’s trajectory. Finally, to ensure precise semantic matching, a fine-grained interaction module aligns these dynamic video tokens with textual details. Extensive experiments on MSRVTT, ActivityNet, and DiDeMo demonstrate that TRFG achieves state-of-the-art performance across multiple backbones and retrieval tasks. Ablation studies confirm the complementarity and generalizability of both modules, underscoring the importance of explicit temporal modeling and fine-grained interaction in bridging the modality gap.

Online platforms’ openness makes Recommender Systems (RSs) susceptible to data poisoning attacks, where malicious user profiles are injected into the training dataset to distort recommendation outcomes. However, existing poisoning attack methods often struggle to achieve an optimal effectiveness on both imperceptibility and aggressiveness. To address this issue, we propose a novel poisoning attack method for RSs, named MDPAttack, which consists of three key modules, each focusing on imperceptibility and aggressiveness. Specifically, we first train a Multinomial Diffusion Model (MDM) to model discrete rating data, effectively minimizing information loss during data processing and thereby enhancing the imperceptibility of the generated profiles. Then, we combine the influence function with the Fast Gradient Sign Method (FGSM) to iteratively improve the aggressiveness of poisoning profiles by leveraging template profiles. Finally, these two properties are seamlessly integrated within the MDPAttack framework. Extensive experiments on both classic and modern deep learning-based RSs demonstrate that MDPAttack generates highly imperceptible profiles while maintaining attack performance comparable to state-of-the-art methods.

Music recommender systems play a critical role in music streaming platforms by providing users with music that they are likely to enjoy. Recent studies have shown that user emotions can influence users’ preferences for music moods. However, existing emotion-aware music recommender systems (EMRSs) explicitly or implicitly assume that users’ actual emotional states expressed through identical emotional words are homogeneous. They also assume that users’ music mood preferences are homogeneous under the same emotional state. In this article, we propose four types of heterogeneity that an EMRS should account for: emotion heterogeneity across users, emotion heterogeneity within a user, music mood preference heterogeneity across users, and music mood preference heterogeneity within a user. We further propose a Heterogeneity-aware Deep Bayesian Network (HDBN) to model these assumptions. The HDBN mimics a user’s decision process of choosing music with four components: personalized prior user emotion distribution modeling, posterior user emotion distribution modeling, user grouping, and Bayesian neural network-based music mood preference prediction. We constructed two datasets, called EmoMusicLJ and EmoMusicLJ-small, to validate our method. Extensive experiments demonstrate that our method significantly outperforms baseline approaches on metrics of HR, Precision, NDCG, and MRR. Ablation studies and case studies further validate the effectiveness of our HDBN. The source code and datasets are available at https://github.com/jingrk/HDBN .

The rapid expansion of Location-Based Social Networks (LBSNs) has highlighted the importance of effective next Point-of-Interest (POI) recommendations, which leverage historical check-in data to predict users' next POIs to visit. Traditional centralized deep neural networks (DNNs) offer impressive POI recommendation performance but face challenges due to privacy concerns and limited timeliness. In response, on-device POI recommendations have been introduced, utilizing federated learning (FL) and decentralized approaches to ensure privacy and recommendation timeliness. However, these methods often suffer from computational strain on devices and struggle to adapt to new users and regions. This paper introduces a novel collaborative learning framework, Diffusion-Based Cloud-Edge-Device Collaborative Learning for Next POI Recommendations (DCPR), leveraging the diffusion model known for its success across various domains. DCPR operates with a cloud-edge-device architecture to offer region-specific and highly personalized POI recommendations while reducing on-device computational burdens. DCPR minimizes on-device computational demands through a unique blend of global and local learning processes. Our evaluation with two real-world datasets demonstrates DCPR's superior performance in recommendation accuracy, efficiency, and adaptability to new users and regions, marking a significant step forward in on-device POI recommendation technology.

Graph neural networks (GNNs) are currently one of the most performant and versatile collaborative filtering methods. Meanwhile, like in traditional collaborative filtering, owing to the use of an embedding table to represent each user/item entity as a distinct vector, GNN-based recommenders have inherited its long-standing defect of parameter inefficiency. As a common practice for scalable embeddings, parameter sharing enables the use of fewer embedding vectors (which we term meta-embeddings), where each entity is represented by a unique combination of meta-embeddings instead. When assigning meta-embeddings, most existing methods are a heuristically designed, predefined mapping from each user/item entity's ID to the corresponding meta-embedding indexes (e.g., double hashing), thus simplifying the optimization problem into learning only the meta-embeddings. However, in the context of GNN-based collaborative filtering, such a fixed mapping omits the semantic correlations between entities that are evident in the user-item interaction graph, leading to suboptimal recommendation performance. To this end, we propose Lightweight Embeddings for Graph Collaborative Filtering (LEGCF), a parameter-efficient embedding framework dedicated to GNN-based recommenders. LEGCF innovatively introduces an assignment matrix as an additional learnable component on top of meta-embeddings. To jointly optimize these two heavily entangled components, aside from learning the meta-embeddings by minimizing the recommendation loss, LEGCF further performs efficient assignment update by enforcing a novel semantic similarity constraint and finding its closed-form solution based on matrix pseudo-inverse. The meta-embeddings and assignment matrix are alternately updated, where the latter is sparsified on the fly to ensure negligible storage overhead. Extensive experiments on three benchmark datasets have verified LEGCF's smallest trade-off between size and performance, with consistent accuracy gain over state-of-the-art baselines. The codebase of LEGCF is available in https://github.com/xurong-liang/LEGCF.

Text Classification (TC) is a fundamental task in the information retrieval community. Nowadays, the mainstay TC methods are built on the deep neural networks, which can learn much more discriminative text features than the traditional shallow learning methods. Among existing deep TC methods, the ones based on Graph Neural Network (GNN) have attracted more attention due to the superior performance. Technically, the GNN-based TC methods mainly transform the full training dataset to a graph of texts; however, they often neglect the dependency between words, so as to miss potential semantic information of texts, which may be significant to exactly represent them. To solve the aforementioned problem, we generate graphs of words instead, so as to capture the dependency information of words. Specifically, each text is translated into a graph of words, where neighboring words are linked. We learn the node features of words by a GNN-like procedure and then aggregate them as the graph feature to represent the current text. To further improve the text representations, we suggest a contrastive learning regularization term. Specifically, we generate two augmented text graphs for each original text graph, we constrain the representations of the two augmented graphs from the same text close and the ones from different texts far away. We propose various techniques to generate the augmented graphs. Upon those ideas, we develop a novel deep TC model, namely Text-level Graph Networks with Contrastive Learning (TGN cl ). We conduct a number of experiments to evaluate the proposed TGN cl model. The empirical results demonstrate that TGN cl can outperform the existing state-of-the-art TC models.

As a step beyond traditional personalized recommendation, group recommendation is the task of suggesting items that can satisfy a group of users. In group recommendation, the core is to design preference aggregation functions to obtain a quality summary of all group members' preferences. Such user and group preferences are commonly represented as points in the vector space (i.e., embeddings), where multiple user embeddings are compressed into one to facilitate ranking for group-item pairs. However, the resulted group representations, as points, lack adequate flexibility and capacity to account for the multi-faceted user preferences. Also, the point embedding-based preference aggregation is a less faithful reflection of a group's decision-making process, where all users have to agree on a certain value in each embedding dimension instead of a negotiable interval. In this paper, we propose a novel representation of groups via the notion of hypercubes, which are subspaces containing innumerable points in the vector space. Specifically, we design the hypercube recommender (CubeRec) to adaptively learn group hypercubes from user embeddings with minimal information loss during preference aggregation, and to leverage a revamped distance metric to measure the affinity between group hypercubes and item points. Moreover, to counteract the long-standing issue of data sparsity in group recommendation, we make full use of the geometric expressiveness of hypercubes and innovatively incorporate self-supervision by intersecting two groups. Experiments on four real-world datasets have validated the superiority of CubeRec over state-of-the-art baselines.

With the rapid development of online social recommendation system, substantial methods have been proposed. Unlike traditional recommendation system, social recommendation performs by integrating social relationship features, where there are two major challenges, i.e., early summarization and data sparsity. Thus far, they have not been solved effectively. In this article, we propose a novel social recommendation approach, namely Multi-Graph Heterogeneous Interaction Fusion (MG-HIF), to solve these two problems. Our basic idea is to fuse heterogeneous interaction features from multi-graphs, i.e., user–item bipartite graph and social relation network, to improve the vertex representation learning. A meta-path cross-fusion model is proposed to fuse multi-hop heterogeneous interaction features via discrete cross-correlations. Based on that, a social relation GAN is developed to explore latent friendships of each user. We further fuse representations from two graphs by a novel multi-graph information fusion strategy with attention mechanism. To the best of our knowledge, this is the first work to combine meta-path with social relation representation. To evaluate the performance of MG-HIF, we compare MG-HIF with seven states of the art over four benchmark datasets. The experimental results show that MG-HIF achieves better performance.

Real-world data in emerging applications may suffer from highly-skewed class imbalanced distribution, however how to deal with this kind of problem appropriately through deep learning needs further investigation. In this paper, we mainly propose a novel cross-entropy based loss function, referred to as Additive Scale Loss (ASL), for deep representation learning and imbalanced classification. To deal with the class imbalanced problem, ASL aims at increasing the loss in case of misclassification, which can avoid the superimposed loss values caused by the large amount of easily classified data in the unbalanced database to dominate the loss value of misclassified data. Moreover, in real-world applications, one data source may be used for multiple scenarios, such as classification and embedding learning, however training two separable models to handle these problems is costly, especially in deep learning area. To tackle this issue, we present and integrate a discriminative inter-class separation term into ASL, and propose a discriminative ASL (D-ASL), which can not only improve the classification performance, but also obtain discriminative representations simultaneously. The discriminative inter-class separation term is general, and can be easily integrated to other loss functions, such as CE and FL, as the byproducts. Finally, a new deep convolutional neural network equipped with D-ASL and a fully-connected (FC) layer is proposed, which can classify the imbalanced image data and obtain the discriminative representations at the same time. Extensive experimental results verified the superior performance of our method.

In today's context, deploying data-driven services like recommendation on edge devices instead of cloud servers becomes increasingly attractive due to privacy and network latency concerns. A common practice in building compact on-device recommender systems is to compress their embeddings which are normally the cause of excessive parameterization. However, despite the vast variety of devices and their associated memory constraints, existing memory-efficient recommender systems are only specialized for a fixed memory budget in every design and training life cycle, where a new model has to be retrained to obtain the optimal performance while adapting to a smaller/larger memory budget. In this paper, we present a novel lightweight recommendation paradigm that allows a well-trained recommender to be customized for arbitrary device-specific memory constraints without retraining. The core idea is to compose elastic embeddings for each item, where an elastic embedding is the concatenation of a set of embedding blocks that are carefully chosen by an automated search function. Correspondingly, we propose an innovative approach, namely recommendation with universally learned elastic embeddings (RULE). To ensure the expressiveness of all candidate embedding blocks, RULE enforces a diversity-driven regularization when learning different embedding blocks. Then, a performance estimator-based evolutionary search function is designed, allowing for efficient specialization of elastic embeddings under any memory constraint for on-device recommendation. Extensive experiments on real-world datasets reveal the superior performance of RULE under tight memory budgets.

Dictionary learning (DL) is powerful for representation learning, while it fails to capture the deep hierarchical information hidden in data. In this paper, we propose a new generalized end-to-end mulita-layer representation learning architecture referred to as Multi-layer Dictionary Pair Learning Network (MDPL-net) for the deep sparse and hierarchical representation of images. To enable MDPL-net to conduct accurate classification, MDPL-net clearly integrates the skip connection end-to-end network and multi-layer deep sparse dictionary learning into a unified architecture. The representation learning module has several hidden DL blocks, where each hidden DL block has a dictionary pair learning (DPL) layer, a batch-norm layer and an activation function layer, and the DL blocks are connected in a feed-forward manner. To further improve the information flow and maintain the privileged features between different DL blocks, a novel skip dense connectivity pattern is deployed between hidden DL blocks, which can obtain more stable and discriminative features. The DPL layer jointly formulates the discriminative synthesis dictionary and analysis dictionary by minimizing reconstruction error within each batch over the feature maps from front layers. Extensive results on benchmark databases demonstrate the effectiveness of MDPL-net for discriminative representation and robust image classification.

As a fundamental yet significant process in personalized recommendation, candidate generation and suggestion effectively help users spot the most suitable items for them. Consequently, identifying substitutable items that are interchangeable opens up new opportunities to refine the quality of generated candidates. When a user is browsing a specific type of product (e.g., a laptop) to buy, the accurate recommendation of substitutes (e.g., better equipped laptops) can offer the user more suitable options to choose from, thus substantially increasing the chance of a successful purchase. However, existing methods merely treat this problem as mining pairwise item relationships without the consideration of users' personal preferences. Moreover, the substitutable relationships are implicitly identified through the learned latent representations of items, leading to uninterpretable recommendation results.

Video recommendation has become an essential way of helping people explore the massive videos and discover the ones that may be of interest to them. In the existing video recommender systems, the models make the recommendations based on the user-video interactions and single specific content features. When the specific content features are unavailable, the performance of the existing models will seriously deteriorate. Inspired by the fact that rich contents (e.g., text, audio, motion, and so on) exist in videos, in this paper, we explore how to use these rich contents to overcome the limitations caused by the unavailability of the specific ones. Specifically, we propose a novel general framework that incorporates arbitrary single content feature with user-video interactions, named as collaborative embedding regression (CER) model, to make effective video recommendation in both in-matrix and out-of-matrix scenarios. Our extensive experiments on two real-world large-scale datasets show that CER beats the existing recommender models with any single content feature and is more time efficient. In addition, we propose a priority-based late fusion (PRI) method to gain the benefit brought by the integrating the multiple content features. The corresponding experiment shows that PRI brings real performance improvement to the baseline and outperforms the existing fusion methods.

The top- N recommendation problem has been studied extensively. Item-based collaborative filtering recommendation algorithms show promising results for the problem. They predict a user’s preferences by estimating similarities between a target and user-rated items. Top- N recommendation remains a challenging task in scenarios where there is a lack of preference history for new items. Feature-based Similarity Models (FSMs) address this particular problem by extending item-based collaborative filtering by estimating similarity functions of item features. The quality of the estimated similarity function determines the accuracy of the recommendation. However, existing FSMs only estimate global similarity functions; i.e., they estimate using preference information across all users. Moreover, the estimated similarity functions are linear ; hence, they may fail to capture the complex structure underlying item features. In this article, we propose to improve FSMs by estimating local similarity functions, where each function is estimated for a subset of like-minded users. To capture global preference patterns, we extend the global similarity function from linear to nonlinear, based on the effectiveness of variational autoencoders. We propose a Bayesian generative model, called the Local Variational Feature-based Similarity Model, to encapsulate local and global similarity functions. We present a variational Expectation Minimization algorithm for efficient approximate inference. Extensive experiments on a large number of real-world datasets demonstrate the effectiveness of our proposed model.

Top- N recommendations have been studied extensively. Promising results have been achieved by recent item-based collaborative filtering (ICF) methods. The key to ICF lies in the estimation of item similarities. Observing the block-diagonal structure of the item similarities in practice, we propose a block-diagonal regularization (BDR) over item similarities for ICF. The intuitions behind BDR are as follows: (1) with BDR, item clustering is embedded into the learning of ICF methods; (2) BDR induces sparsity of item similarities, which guarantees recommendation efficiency; and (3) BDR captures in-block transitivity to overcome rating sparsity. By regularizing the item similarity matrix of item similarity models with BDR, we obtain a block-aware item similarity model. Our experimental evaluations on a large number of datasets show that the block-diagonal structure is crucial to the performance of top- N recommendation.

In this paper, we propose a robust representation learning model called Adaptive Structure-constrained Low-Rank Coding (AS-LRC) for the latent representation of data. To recover the underlying subspaces more accurately, AS-LRC seamlessly integrates an adaptive weighting based block-diagonal structure-constrained low-rank representation and the group sparse salient feature extraction into a unified framework. Specifically, AS-LRC performs the latent decomposition of given data into a low-rank reconstruction by a block-diagonal codes matrix, a group sparse locality-adaptive salient feature part and a sparse error part. To enforce the block-diagonal structures adaptive to different real datasets for the low-rank recovery, AS-LRC clearly computes an auto-weighting matrix based on the locality-adaptive features and multiplies by the low-rank coefficients for direct minimization at the same time. This encourages the codes to be block-diagonal and can avoid the tricky issue of choosing optimal neighborhood size or kernel width for the weight assignment, suffered in most local geometrical structures-preserving low-rank coding methods. In addition, our AS-LRC selects the L2, 1-norm on the projection for extracting group sparse features rather than learning low-rank features by Nuclear-norm regularization, which can make learnt features robust to noise and outliers in samples, and can also make the feature coding process efficient. Extensive visualizations and numerical results demonstrate the effectiveness of our AS-LRC for image representation and recovery.

In this paper, we extend the popular dictionary pair learning (DPL) into the scenario of twin-projective latent flexible DPL under a structured twin-incoherence. Technically, a novel framework called Twin-Projective Latent Flexible DPL (TP-DPL) is proposed, which minimizes the twin-incoherence constrained flexibly-relaxed reconstruction error to avoid the possible over-fitting issue and produce accurate reconstruction. In this setting, TP-DPL integrates the twin-incoherence based latent flexible DPL and the joint embedding of codes as well as salient features by twin-projection into a unified model in an adaptive neighborhood-preserving manner. Therefore, TP-DPL can unify the procedures of salient feature representation and classification. The twin-incoherence constraint on coefficients and features can explicitly ensure high intra-class compactness and inter-class separation over them. TP-DPL also integrates the adaptive weighting to preserve local neighborhood of both coefficients and salient features within each class explicitly. For efficiency, TP-DPL selects the Frobenius-norm and abandons the costly l0/l1-norm for group sparse representation. Another byproduct is that TP-DPL can directly apply the class-specific twin-projective reconstruction residual to compute the label of data. Extensive results on public databases show that TP-DPL can deliver the state-of-the-art performance.

Factorization Machines (FMs) are widely used for feature-based collaborative filtering tasks, as they are very effective at modeling feature interactions. Existing FM-based methods usually take all feature interactions into account, which is unreasonable because not all feature interactions are helpful: incorporating useless feature interactions will introduce noise and degrade the recommendation performance. Recently, methods that perform Feature Interaction Selection (FIS) have attracted attention because of their effectiveness at filtering out useless feature interactions. However, they assume that all users share the same feature interactions, which is not necessarily true, especially for collaborative filtering tasks. In this work, we address this issue and study Personalized Feature Interaction Selection (P-FIS) by proposing a Bayesian Personalized Feature Interaction Selection (BP-FIS) mechanism under the Bayesian Variable Selection (BVS) theory. Specifically, we first introduce interaction selection variables with hereditary spike and slab priors for P-FIS. Then, we form a Bayesian generative model and derive the Evidence Lower Bound (ELBO), which can be optimized by an efficient Stochastic Gradient Variational Bayes (SGVB) method to learn the parameters. Finally, because BP-FIS can be seamlessly integrated with different variants of FMs, we implement two FM variants under the proposed BP-FIS. We carry out experiments on three benchmark datasets. The empirical results demonstrate the effectiveness of BP-FIS for selecting personalized interactions and improving the recommendation performance.

Uncertainty is intrinsic to a wide spectrum of real-life applications, which inevitably applies to graph data. Representative uncertain graphs are seen in bio-informatics, social networks, etc. This paper motivates the problem of frequent subgraph mining on single uncertain graphs, and investigates two different - probabilistic and expected - semantics in terms of support definitions. First, we present an enumeration-evaluation algorithm to solve the problem under probabilistic semantics. By showing the support computation under probabilistic semantics is #P-complete, we develop an approximation algorithm with accuracy guarantee for efficient problem-solving. To enhance the solution, we devise computation sharing techniques to achieve better mining performance. Afterwards, the algorithm is extended in a similar flavor to handle the problem under expected semantics, where checkpoint-based pruning and validation techniques are integrated. Experiment results on real-life datasets confirm the practical usability of the mining algorithms.

With the rapid rise of various e-commerce and social network platforms, users are generating large amounts of heterogeneous behavior data, such as purchasehistory, adding-to-favorite, adding-to-cart and click activities, and this kind of user behavior data is usually binary, only reflecting a user's action or inaction (i.e., implicit feedback data). Tensor factorization is a promising means of modeling heterogeneous user behaviors by distinguishing different behavior types. However, ambiguity arises in the interpretation of the unobserved user behavior records that mix both real negative examples and potential positive examples. Existing tensor factorization models either ignore unobserved examples or treat all of them as negative examples, leading to either poor prediction performance or huge computation cost. In addition, the distribution of positive examples w.r.t. behavior types is heavily skewed. Existing tensor factorization models would bias towards the type of behaviors with a large number of positive examples. In this paper, we propose a scalable probabilistic tensor factorization model (SPTF) for heterogeneous behavior data and develop a novel negative sampling technique to optimize SPTF by leveraging both observed and unobserved examples with much lower computational costs and higher modeling accuracy. To overcome the issue of the heavy skewness of the behavior data distribution, we propose a novel adaptive ranking-based positive sampling approach to speed up the model convergence and improve the prediction accuracy for sparse behavior types. Our proposed model optimization techniques enable SPTF to be scalable to large-scale behavior datasets. Extensive experiments have been conducted on a large-scale e-commerce dataset, and the experimental results show the superiority of our proposed SPTF model in terms of prediction accuracy and scalability.

Uncertainty is intrinsic to a wide spectrum of real-life applications, which inevitably applies to graph data. Representative uncertain graphs are seen in bio-informatics, social networks, etc. This paper motivates the problem of frequent subgraph mining on single uncertain graphs. We present an enumeration-evaluation algorithm to solve the problem. By showing support computation on an uncertain graph is #P-hard, we develop an approximation algorithm with accuracy guarantee for this purpose. To enhance the solution, we devise optimization techniques to achieve better mining performance. Experiment results on real-life data confirm the usability of the algorithm.

Hashing has gained considerable attention on large-scale similarity search, due to its enjoyable efficiency and low storage cost. In this paper, we study the problem of learning hash functions in the context of multi-modal data for cross-modal similarity search. Notwithstanding the progress achieved by existing methods, they essentially learn only one common hamming space, where data objects from all modalities are mapped to conduct similarity search. However, such method is unable to well characterize the flexible and discriminative local (neighborhood) structure in all modalities simultaneously, hindering them to achieve better performance. Bearing such stand-out limitation, we propose to learn heterogeneous hamming spaces with each preserving the local structure of data objects from an individual modality. Then, a novel method to learning bridging mapping for cross-modal hashing, named LBMCH, is proposed to characterize the cross-modal semantic correspondence by seamlessly connecting these distinct hamming spaces. Meanwhile, the local structure of each data object in a modality is preserved by constructing an anchor based representation, enabling LBMCH to characterize a linear complexity w.r.t the size of training set. The efficacy of LBMCH is experimentally validated against real-world cross-modal datasets.

Many real-world objects described by multiple attributes or features can be decomposed as multiple "views" (e.g., an image can be described by a color view or a shape view), which often provides complementary information to each other. Learning a metric (similarity measures) for multi-view data is primary due to its wide applications in practices. However, leveraging multi-view information to produce a good metric is a great challenge and existing techniques are concerned with pairwise similarities, leading to undesirable fusion metric and high computational complexity. In this paper, we propose a novel Metric Fusion technique via cross-view graph Random Walk, named MFRW, regarding a multi-view based similarity graphs (with each similarity graph constructed under each view). Instead of using pairwise similarities, we seek a high-order metric yielded by graph random walks over constructed similarity graphs. Observing that ``outlier views" may exist in the fusion process, we incorporate the coefficient matrices representing the correlation strength between any two views into MFRW, named WMFRW. The principle of \textsf{WMFRW} is implemented by exploring the ``common latent structure" between views. The empirical studies conducted on real-world databases demonstrate that our approach outperforms the state-of-the-art competitors in terms of effectiveness and efficiency.