Real-world scientific applications frequently encounter incomplete observational data due to sensor limitations, geographic constraints, or measurement costs. Although neural operators significantly advanced PDE solving in terms of computational efficiency and accuracy, their underlying assumption of fully-observed spatial inputs severely restricts applicability in real-world application. We introduce the first systematic framework for learning neural operators from partial observation. We identify and formalize two fundamental obstacles: (i) the supervision gap in unobserved regions that prevents effective learning of physical correlations, and (ii) the dynamic spatial mismatch between incomplete inputs and complete solution fields. Specifically, our proposed LANO (Latent Autoregressive Neural Operator) introduces two novel components designed explicitly to address the core difficulties of partial observations: (i) a mask-to-predict training strategy that creates artificial supervision by strategically masking observed regions, and (ii) a Physics-Aware Latent Propagator that reconstructs solutions through boundary-first autoregressive generation in latent space. Additionally, we develop POBench-PDE, a dedicated and comprehensive benchmark designed specifically for evaluating neural operators under partial observation conditions across three PDE-governed tasks. LANO achieves state-of-the-art performance with relative error reductions ranging from eighteen to sixty-nine percent across all benchmarks under patch-wise missingness with missing rates below fifty percent, including real-world climate prediction. Our approach effectively addresses practical scenarios with missing rates of up to seventy-five percent, to some extent bridging the existing gap between idealized research settings and the complexities of real-world scientific computing.

Meta-learning for Bayesian optimization accelerates optimization by leveraging knowledge from previous tasks, but existing methods optimize for average performance and fail on challenging outlier tasks critical in practice. These limitations become particularly severe when target tasks exhibit distribution shifts or when optimization budgets are limited in real-world applications. We introduce MetaGameBO, a hierarchical game-theoretic framework that formulates meta-learning as robust optimization through CVaR-based task selection and diversity-aware sample learning. Our approach incorporates uncertainty-aware adaptation via probabilistic embeddings and Thompson sampling for robust generalization to out-of-distribution targets. We establish theoretical guarantees including convergence to game-theoretic equilibria and improved sample complexity, and demonstrate substantial improvements with 95.7% reduction in average loss and 88.6% lower tail risk compared to state-of-the-art methods on challenging tasks and distribution shifts.

Neural processes (NPs) are a promising paradigm to enable skill transfer learning across tasks with the aid of the distribution of functions. The previous NPs employ the empirical risk minimization principle in optimization. However, the fast adaption ability to different tasks can vary widely, and the worst fast adaptation can be catastrophic in risk-sensitive tasks. To achieve robust neural processes modeling, we consider the problem of training models in a risk-averse manner, which can control the worst fast adaption cases at a certain probabilistic level. By transferring the risk minimization problem to a two-level finite sum minimax optimization problem, we can easily solve it via a double-looped stochastic mirror prox algorithm with a task-aware variance reduction mechanism via sampling samples across all tasks. The mirror prox technique ensures better handling of complex constraint sets and non-Euclidean geometries, making the optimization adaptable to various tasks. The final solution, by aggregating prox points with the adaptive learning rates, enables a stable and high-quality output. The proposed learning strategy can work with various NPs flexibly and achieves less biased approximation with a theoretical guarantee. To illustrate the superiority of the proposed model, we perform experiments on both synthetic and real-world data, and the results demonstrate that our approach not only helps to achieve more accurate performance but also improves model robustness.

Continuous-time dynamic modeling of irregular time series is crucial for capturing the continuous evolution and complex correlations within the data. Traditional methods, such as Recurrent Neural Network (RNN)-based models, often require discretization of the time series, which may lead to information loss or insufficient capture of dynamic relationships between data points. Although Neural Ordinary Differential Equations (Neural ODEs) and their variants have shown some potential in handling irregular time series, they often struggle to effectively address the stochasticity in the data and capture complex dynamic relationships within the sequences. To address these issues, we propose SDEFormer, a continuous-time dynamic Transformer model based on Neural Stochastic Differential Equations (Neural SDEs). SDEFormer combines the continuous-time modeling capabilities of Neural SDEs with the sequence modeling strengths of Transformers, enabling it to capture both deterministic and stochastic components in complex continuous-time dynamics. We mathematically describe the expressive power of SDEFormer and address the stability issues inherent in SDEs by designing three stable SDE-based variants. Experiments on multiple irregular time series classification and event prediction tasks demonstrate that SDEFormer exhibits stronger robustness when dealing with sparse and irregular data, outperforming traditional RNNs, ODE methods, and some Transformer variants in terms of prediction accuracy and stability. By integrating continuous-time modeling with attention mechanisms, SDEFormer effectively captures complex temporal dependencies and system behaviors in irregular time series, showcasing its powerful capabilities in irregular time series analysis.

Existing work in physical-informed machine learning (PIML) has shown that data-driven learning of solution operators can provide a fast approximate alternative to classical numerical ordinary/partial differential equations (ODEs/PDEs) solvers. Of these, Neural Operators (NOs) have emerged as particularly promising. However, a key challenge in the field of NOs lies in developing methods that can effectively handle out-of-distribution (OOD) forecasting problems. Such problems involve the ability to adaptively learn from observations of the same dynamical system governed by ODEs/PDEs, where the underlying parameters are unknown and vary across instances. These tasks further require precise predictions even when faced with initial conditions and PDEs/ODEs parameters outside the training distribution. In this study, we consider the problem of training models in a risk-reverse manner. We introduce a risk-aware framework aimed at enhancing the OOD robustness of NOs by stochastically optimizing the conditional value-at-risk (CVAR) of a loss distribution. Through experiments on different distinct OOD tasks, our approach demonstrates a significant performance improvement over existing advanced NOs.

Neural Processes (NPs) combine the adaptability of neural networks with the efficiency of meta-learning, offering a powerful framework for modeling stochastic processes. However, existing methods focus on empirical performance while lacking a rigorous theoretical understanding of generalization. To address this, we propose an information-theoretic framework to analyze the generalization bounds of NPs, introducing dynamical stability regularization to minimize sharpness and improve optimization dynamics. Additionally, we show how noise-injected parameter updates complement this regularization. The proposed approach, applicable to a wide range of NP models, is validated through experiments on classic benchmarks, including 1D regression, image completion, Bayesian optimization, and contextual bandits. The results demonstrate tighter generalization bounds and superior predictive performance, establishing a principled foundation for advancing generalizable NP models.

Most works of interpretable neural networks strive for learning the semantics concepts merely from single modal information such as images. However, humans usually learn semantic concepts from multiple modalities and the semantics is encoded by the brain from fused multi-modal information. Inspired by cognitive science and vision-language learning, we propose a Prototype-Concept Alignment Network (ProCoNet) for learning visual prototypes under the guidance of textual concepts. In the ProCoNet, we have designed a visual encoder to decompose the input image into regional features of prototypes, while also developing a prompt generation strategy that incorporates in-context learning to prompt large language models to generate textual concepts. To align visual prototypes with textual concepts, we leverage the multimodal space provided by the pre-trained CLIP as a bridge. Specifically, the regional features from the vision space and the cropped regions of prototypes encoded by CLIP reside on different but semantically highly correlated manifolds, i.e. follow a multi-manifold distribution. We transform the multi-manifold distribution alignment problem into optimizing the projection matrix by Cayley transform on the Stiefel manifold. Through the learned projection matrix, visual prototypes can be projected into the multimodal space to align with semantically similar textual concept features encoded by CLIP. We conducted two case studies on the CUB-200-2011 and Oxford Flower dataset. Our experiments show that the ProCoNet provides higher accuracy and better interpretability compared to the single-modality interpretable model. Furthermore, ProCoNet offers a level of interpretability not previously available in other interpretable methods.

When humans explain their reasoning, such as their classification decisions, they often break down an image into parts and highlight the evidence from those parts to support the concepts they have in mind. Drawing inspiration from this cognitive process, several self-explaining models have been proposed to explain predictions by part-level concepts. However, these models can be limited by their structure and difficulty in determining the effect of individual parts on the output category. To address these challenges, we introduce a self-explaining architecture that uses a plug-in transparent embedding space (TesNet) to connect high-level input patches (e.g. feature maps or tokens) with output categories. The transparent embedding space is spanned by basis concepts and constructed on the Grassmann manifold. The basis concepts are enforced to be category-aware, and within-category concepts are orthogonal to each other, ensuring the embedding space is disentangled. To reduce concept redundancy and restore the concept space structure, we introduce two concept pruning methods and a new re-training strategy to build a slimming transparent embedding space. We verify the scalability of TesNet through experiments on deep networks such as VGG, ResNet, DenseNet, and Vision Transformer. Additionally, we design several metrics for self-explaining models to quantify interpretability and compare them with state-of-the-art self-explaining methods. Our experiments demonstrate that TesNet is much more effective for classification tasks, providing better interpretability on predictions and improving final accuracy.

As an important branch of unsupervised learning methods, clustering makes a wide contribution in the area of data mining. It is well known that capturing the group-discriminative properties of each sample for clustering is crucial. Among them, deep clustering delivers promising results due to the strong representational power of neural networks. However, most of them adopt sample-level learning strategies, and the standalone data point barely captures its holistic cluster’s context and may undergo sub-optimal cluster assignment. To tackle this issue, we propose a Structure-driven Representation Learning (SRL) method by introducing latent structure information into the representation learning process at both the local and global levels. Specifically, a local-structure-driven sample representation strategy is proposed to approximate the estimation of data distribution, which models the neighborhood distribution of samples with potential structure information and exploits statistical dependencies between them to improve cluster consistency. A global-structure-driven cluster representation strategy is designed, where the context of each cluster is sufficiently encoded according to its samples (exemplar-theory) and corresponding prototype (prototype-theory). In this case, each cluster can only be related to its most similar samples, and different clusters are separated as much as possible. These two models are seamlessly combined into a joint optimization problem, which can be efficiently solved. Experiments on six widely-used datasets demonstrate the superiority of SRL over state-of-the-art clustering methods.

User preferences behind users' decision-making processes are highly diverse and may range from lower-level concepts with more specific intentions and higher-level concepts with more general intentions. In this case, user preferences tend to be expressed hierarchically. However, learning such intentions with different levels from user behaviors is challenging, and remains largely neglected by the existing literature. Meanwhile, user behavior data tends to be sparse because of the limited user response and the vast combinations of users and items, which results in cold-start problems with unclear user intentions. In this paper, we propose a mixture intention neural stochastic process (MINSP), a new view of the stochastic processes family using a general meta-learning mechanism and mixture strategy for robust recommendation with hierarchical preferences modeling. By considering the recommendation process for each user as a stochastic process, MINSP defines distributions over functions and is capable of rapid adaptation to different users. To capture the user's intention on different levels, an iterative additive algorithm is proposed that minimizes the approximation error by backfitting the residuals of previous approximations. In this case, the induced tree intention hierarchies serve as an aggregated structured representation of the whole preference, summarizing the gist for convenient navigation and better generalization. Furthermore, we theoretically analyze the generalization error bound of the proposed MINSP to guarantee the model performance. Empirical results show that our approach can achieve substantial improvement over the state-of-the-art baselines in terms of recommendation performance, and obtain an interpretable hierarchical intention structure.

Keyphrase extraction aims to provide readers with high-level information about the central ideas or important topics described in a given source text. Recent advances in embedding-based models have made remarkable progress on unsupervised keyphrase extraction, demonstrated through improved quality metrics such as F1-score. However, the diversity in the keyphrase extraction task needs to be addressed. In this paper, we focus on diverse keyphrase extraction, which entails extracting keyphrases that cover different central information or essential topics in the document. To achieve this goal, we propose a re-ranking-based approach that employs determinantal point processes utilizing BERT as kernels, which we call DiversityRank. Specifically, DiversityRank jointly considers phrase-document relevance and cross-phrase similarities to select candidate keyphrases that are document-relevant and diverse. Results demonstrate that our re-ranking strategy outperforms the state-of-the-art unsupervised keyphrase extraction baselines on three benchmark datasets.

=User cold-start recommendation is a well-known challenge in current recommender systems. The cause is that the number of user interactions is too few to accurately estimate user preferences. Furthermore, the uncertainty of user interactions intensifies along with the number of user interactions decreasing. Although existing meta-learning based models with globally sharing knowledge show good performance in most cold-start scenarios, the ability of handling challenges on intention importance and prediction uncertainty is missing: (1) Intra-user uncertainty. When estimating user preferences (reflected in the user's latent representation), each of user interactions is independently considered in the form of user-item pair, which cannot capture the correlation between user interactions, as well as considering the global intent under user interactions. (2) Inter-user importance. During the model training, all users are treated as equally important, which cannot distinguish the contribution of users in the model training process. Assigning the same weight to all users may lead to users with high uncertainty incorrectly guiding the model learning in the early stage of training. To tackle the above challenges, in this paper, we focus on modeling user preference as a weighted distribution over functions (WDoF) for user cold-start recommendation, which not only models the intra-user uncertainty through neural processes with Multinomial likelihood but also considers the importance of different users with curriculum learning during the model training process. Furthermore, we provide a theoretical explanation that why the proposed model performs better than regular neural processes based recommendation methods. Experiments on four real-world datasets demonstrate the effectiveness of the proposed model over several state-of-the-art cold-start recommendation methods.

Given the exponential growth in the number of documents on the web in recent years, there is an increasing demand for accurate models to extract keyphrases from such documents.Keyphrase extraction is the task of automatically identifying representative keyphrases from the source document.Typically, candidate keyphrases exhibit latent hierarchical structures embedded with intricate syntactic and semantic information.Moreover, the relationships between candidate keyphrases and the document also form hierarchical structures.Therefore, it is essential to consider these latent hierarchical structures when extracting keyphrases.However, many recent unsupervised keyphrase extraction models overlook this aspect, resulting in incorrect keyphrase extraction.In this paper, we address this issue by proposing a new hyperbolic ranking model (HyperRank).HyperRank is designed to jointly model global and local context information for estimating the importance of each candidate keyphrase within the hyperbolic space, enabling accurate keyphrase extraction.Experimental results demonstrate that HyperRank significantly outperforms recent state-of-the-art baselines.

Over-generation errors occur when a keyphrase extraction model correctly determines a candidate keyphrase as a keyphrase because it contains a word that frequently appears in the document but at the same time erroneously outputs other candidates as keyphrases because they contain the same word.To mitigate this issue, we propose a new heterogeneous centrality detection approach (CentralityRank), which extracts keyphrases by simultaneously identifying both implicit and explicit centrality within a heterogeneous graph as the importance score of each candidate.More specifically, Centrali-tyRank detects centrality by taking full advantage of the content within the input document to construct graphs that encompass semantic nodes of varying granularity levels, not limited to just phrases.These additional nodes act as intermediaries between candidate keyphrases, enhancing inter-phrase relevance.Furthermore, we introduce a novel adaptive boundary-aware regularization that can leverage the position information of candidate keyphrases, thus influencing the importance of candidate keyphrases.Extensive experimental results demonstrate the superiority of CentralityRank over recent stateof-the-art unsupervised keyphrase extraction baselines on three benchmark datasets.

The cold-start recommendation has been one of the most central problems in online platforms where new users or items arrive continuously. Although existing meta-learning based models with globally sharing knowledge show good performance in most cold-start scenarios, the ability to handle challenges on intention heterogeneity and prediction uncertainty is missing, and these two challenges are particularly evident in cold-start scenarios with fewer interaction data. To tackle the above challenges, in this paper, we present an uncertainty-aware Stochastic Meta Process with Doubly Intention learning (DISMP) for the cold-start recommendation, which has promising properties in uncertainty quantification. With the aid of the meta-learning stochastic process, DISMP can store general knowledge by capturing the relevance of different user-item pairs in terms of intentions and concepts, which is capable of rapid adaptation to new users and items. Furthermore, intentions with general and specific levels are extracted by doubly distinguishing the role of latent variables, which is able to capture the dependencies across different types of intentions and concepts. Empirical results show that our approach can achieve substantial improvement over the state-of-the-art baselines on cold-start recommendations with different perspectives.

Unlike traditional statistical models depending on hand-specified priors, neural processes (NPs) have recently emerged as a class of powerful neural statistical models that combine the strengths of neural networks and stochastic processes. NPs can define a flexible class of stochastic processes well suited for highly non-trivial functions by encoding contextual knowledge into the function space. However, noisy context points introduce challenges to the algorithmic stability that small changes in training data may significantly change the models and yield lower generalization performance. In this paper, we provide theoretical guidelines for deriving stable solutions with high generalization by introducing the notion of algorithmic stability into NPs, which can be flexible to work with various NPs and achieves less biased approximation with theoretical guarantees. To illustrate the superiority of the proposed model, we perform experiments on both synthetic and real-world data, and the results demonstrate that our approach not only helps to achieve more accurate performance but also improves model robustness.

Discovering hidden pattern from imbalanced data is a critical issue in various real-world applications. Existing classification methods usually suffer from the limitation of data especially for minority classes, and result in unstable prediction and low performance. In this paper, a deep generative classifier is proposed to mitigate this issue via both model perturbation and data perturbation. Specially, the proposed generative classifier is derived from a deep latent variable model where two variables are involved. One variable is to capture the essential information of the original data, denoted as latent codes, which are represented by a probability distribution rather than a single fixed value. The learnt distribution aims to enforce the uncertainty of model and implement model perturbation, thus, lead to stable predictions. The other variable is a prior to latent codes so that the codes are restricted to lie on components in Gaussian Mixture Model. As a confounder affecting generative processes of data (feature/label), the latent variables are supposed to capture the discriminative latent distribution and implement data perturbation. Extensive experiments have been conducted on widely-used real imbalanced image datasets. Experimental results demonstrate the superiority of our proposed model by comparing with popular imbalanced classification baselines on imbalance classification task.

Text classification, is one of the key tasks for representing the semantic information of documents, multi-label text classification (MLTC) is an important branch of it. MLTC aims to tag the most relevant labels for the given document. Compared to the standard multi-class case where each document has only one label, it is considerably more difficulty to annotate new coming documents for multi-label text classification. Furthermore, it also suffers from the challenge of highly skewed long-tailed label distribution. i.e., a few labels are associated with a large number of documents (a.k.a. head labels), while a large fraction of labels are associated with a small number of documents (a.k.a. tail labels). Due to the relative infrequency of tail labels, this leads to an imbalance that biases towards predicting more head labels. As challenging as this task is, it is an essential task to tackle since it represents many real-world cases, such as text retrieval of news. To address the challenge, we propose a Triple Alliance Prototype Orthotist Network (TAPON) to build a generic meta-mapping from few-shot prototypes to many-shot classifier parameters, which aims to promote the generalizability of tail classifiers. To be specific, TAPON is a two-stage method. At the first stage focusing on head labels, TAPON obtains the meta-knowledge between many-shot classifier parameters and few-shot prototype of head labels. Head label classifiers are trained by many-shot documents. Meanwhile, the triple alliance prototype is obtained by adopting an Attentive Prototype with the aid of few-shot documents, label semantic information and label correlation. Additionally, a Prototype Orthotist module is especially designed to capture the meta-knowledge between the many-shot classifier and few-shot prototype. At the second stage of transferring, TAPON aims to transfer the generic meta-mapping from head labels to tail labels. It first uses Attentive Prototype to obtain triple alliance prototype for tail labels, and then uses the meta-knowledge obtained from the first stage to get many-shot classifiers for tail labels. By conducting extensive experiments on four benchmark datasets, we show that the proposed TAPON significantly outperforms other state-of-the-art methods for long-tailed multi-label text classification.

In this paper, we propose Deep amortized Relational Model (DaRM) with group-wise hierarchical generative process for community discovery and link prediction on relational data (e.g., graph, network). It provides an efficient neural relational model architecture by grouping nodes in a group-wise view rather than node-wise or edge-wise view. DaRM simultaneously learns what makes a group, how to divide nodes into groups, and how to adaptively control the number of groups. The dedicated group generative process is able to sufficiently exploit pair-wise or higher-order interactions between data points in both inter-group and intra-group, which is useful to sufficiently mine the hidden structure among data. A series of experiments have been conducted on both synthetic and real-world datasets. The experimental results demonstrated that DaRM can obtain high performance on both community detection and link prediction tasks.

User behavior data in recommendation are driven by the complex interactions of many intentions behind the user's decision making process. However, user behavior data tends to be sparse because of the limited user response and the vase combinations of users and items, which result in unclear user intentions and suffer from cold-start problem. The intentions are highly compound, and may range from high-level ones that govern user's intrinsic interests and realize the underlying reasons behind the user's decision making processes, to low-level one that characterize a user's extrinsic preference when executing intention to specific items. In this paper, we propose an intention neural process model (INP) for user cold-start recommendation (i.e., user with very few historical interactions), a novel extension of the neural stochastic process family using a general meta learning strategy with intrinsic and extrinsic intention learning for robust user preference learning. By regarding the recommendation process for each user as a stochastic process, INP defines distributions over functions, is capable of rapid adaptation to new users. Our approach learns intrinsic intentions by inferring the high-level concepts associated with user interests or purposes, while capturing the target preference of a user by performing self-supervised intention matching between historical items and target items in a disentangled latent space. Extrinsic intentions are learned by simultaneously generating the point-wise implicit feedback data and creates the pair-wise ranking list by sufficient exploiting both interacted and non-interacted items for each user. Empirical results show that our approach can achieve substantial improvement over the state-of-the-art baselines on cold-start recommendation.

Considering the ever-increasing scale of data, which may contain tens of thousands of data points or complicated latent structures, the issue of scalability and algorithmic efficiency becomes of vital importance for clustering. In this paper, we propose cluster-wise amortized mixing coupling processes (AMCP), which is able to achieve efficient amortized clustering in a well-defined non-parametric Bayesian posterior. Specifically, AMCP learns clusters sequentially with the aid of the proposed intra-cluster mixing (IntraCM) and inter-cluster coupling (InterCC) strategies, which investigate the relationship between data points and reference distribution in a linear optimal transport mixing view, and coupling the unassigned set and assigned set to generate new cluster. IntraCM and InterCC avoid pairwise calculation of distances between clusters and reduce the computational complexity from quadratic to linear in the current number of clusters. Furthermore, cluster-wise sequential process is able to improve the quick adaptation ability for the next cluster generation. In this case, AMCP simultaneously learns what makes a cluster, how to group data points into clusters, and how to adaptively control the number of clusters. To illustrate the superiority of the proposed method, we perform experiments on both synthetic data and real-world data in terms of clustering performance and computational efficiency. The source code is available at https://github.com/HuafengHK/AMCP.

Social relations between users have been proven to be a good type of auxiliary information to improve the recommendation performance. However, it is a challenging issue to sufficiently exploit the social relations and correctly determine the user preference from both social and rating information. In this article, we propose a unified Bayesian Additive Matrix Approximation model (BAMA), which takes advantage of rating preference and social network to provide high-quality recommendation. The basic idea of BAMA is to extract social influence from social networks, integrate them to Bayesian additive co-clustering for effectively determining the user clusters and item clusters, and provide an accurate rating prediction. In addition, an efficient algorithm with collapsed Gibbs Sampling is designed to inference the proposed model. A series of experiments were conducted on six real-world social datasets. The results demonstrate the superiority of the proposed BAMA by comparing with the state-of-the-art methods from three views, all users, cold-start users, and users with few social relations. With the aid of social information, furthermore, BAMA has ability to provide the explainable recommendation.

In this paper, we propose Cluster-wise Hierarchical Generative Model for deep amortized clustering (CHiGac). It provides an efficient neural clustering architecture by grouping data points in a cluster-wise view rather than point-wise view. CHiGac simultaneously learns what makes a cluster, how to group data points into clusters, and how to adaptively control the number of clusters. The dedicated cluster generative process is able to sufficiently exploit pair-wise or higher-order interactions between data points in both inter- and intra-cluster, which is useful to sufficiently mine the hidden structure among data. To efficiently minimize the generalized lower bound of CHiGac, we design an Ergodic Amortized Inference (EAI) strategy by considering the average behavior over sequence on an inner variational parameter trajectory, which is theoretically proven to reduce the amortization gap. A series of experiments have been conducted on both synthetic and real-world data. The experimental results demonstrated that CHiGac can efficiently and accurately cluster datasets in terms of both internal and external evaluation metrics (DBI and ACC).

Humans usually explain their reasoning (e.g. classification) by dissecting the image and pointing out the evidence from these parts to the concepts in their minds. Inspired by this cognitive process, several part-level interpretable neural network architectures have been proposed to explain the predictions. However, they suffer from the complex data structure and confusing the effect of the individual part to output category. In this work, an interpretable image recognition deep network is designed by introducing a plug-in transparent embedding space (TesNet) to bridge the high-level input patches (e.g. CNN feature maps) and the out- put categories. This plug-in embedding space is spanned by transparent basis concepts which are constructed on the Grassmann manifold. These basis concepts are enforced to be category-aware and within-category concepts are orthogonal to each other, which makes sure the embedding space is disentangled. Meanwhile, each basis concept can be traced back to the particular image patches, thus they are transparent and friendly to explain the reasoning process. By comparing with state-of-the-art interpretable methods, TesNet is much more beneficial to classification tasks, esp. providing better interpretability on predictions and improve the final accuracy. The code is available at https://github.com/JackeyWang96/TesNet.

User preference modeling in recommendation system aims to improve customer experience through discovering users’ intrinsic preference based on prior user behavior data. This is a challenging issue because user preferences usually have complicated structure, such as inter-user preference similarity and intra-user preference diversity. Among them, inter-user similarity indicates different users may share similar preference, while intra-user diversity indicates one user may have several preferences. In literatures, deep generative models have been successfully applied in recommendation systems due to its flexibility on statistical distributions and strong ability for non-linear representation learning. However, they suffer from the simple generative process when handling complex user preferences. Meanwhile, the latent representations learned by deep generative models are usually entangled, and may range from observed-level ones that dominate the complex correlations between users, to latent-level ones that characterize a user’s preference, which makes the deep model hard to explain and unfriendly for recommendation. Thus, in this paper, we propose an Interpretable Deep Generative Recommendation Model (InDGRM) to characterize inter-user preference similarity and intra-user preference diversity, which will simultaneously disentangle the learned representation from observed-level and latent-level. In InDGRM, the observed-level disentanglement on users is achieved by modeling the user-cluster structure (i.e., inter-user preference similarity) in a rich multimodal space, so that users with similar preferences are assigned into the same cluster. The observed-level disentanglement on items is achieved by modeling the intra-user preference diversity in a prototype learning strategy, where different user intentions are captured by item groups (one group refers to one intention). To promote disentangled latent representations, InDGRM adopts structure and sparsity-inducing penalty and integrates them into the generative procedure, which has ability to enforce each latent factor focus on a limited subset of items (e.g., one item group) and benefit latent-level disentanglement. Meanwhile, it can be efficiently inferred by minimizing its penalized upper bound with the aid of local variational optimization technique. Theoretically, we analyze the generalization error bound of InDGRM to guarantee its performance. A series of experimental results on four widely-used benchmark datasets demonstrates the superiority of InDGRM on recommendation performance and interpretability.

Due to leveraging social relationships between users as well as their past social behavior, social recommendation becomes a core component in recommendation systems. Most existing social recommendation methods only consider direct social relationships among users (e.g., explicit and observed social relations). Recently, researchers proved that indirect social relationships can be effective to improve the recommendation quality when users only have few social connections, because it can identify the user interesting group even though the users have no observed social connection. In the literature, separate two-stage methods are studied, but they cannot explicitly capture the natural relationship between indirect social relations and latent user/item factors. In this paper, the main contribution is to propose a new joint recommendation model taking advantage of the Indirect Social Relations detection and Matrix Factorization collaborative filtering on social network and rating behavior information, which is called as InSRMF. In our work, the user latent factors can simultaneously and seamlessly capture user's personal preferences and social group characteristics. To optimize the InSRMF model, we develop a parallel graph vertex programming algorithm for efficiently handling large scale social recommendation data. Experiments based on four real-world datasets (Ciao, Epinions, Douban and Yelp) are conducted to demonstrate the performance of the proposed model. The experimental results have shown that InSRMF has ability to mine the proper indirect social relations and improve the recommendation performance compared with the testing methods in the literature, especially on the users with few social neighbors, Near-cold-start Users, Pure-cold-start Users and Long-tail Items.

Deep generative model, especially variational auto-encoder (VAE), has been successfully employed by more and more recommendation systems. The reason is that it combines the flexibility of probabilistic generative model with the powerful non-linear feature representation ability of deep neural networks. The existing VAE-based recommendation models are usually proposed under global assumption by incorporating simple priors, e.g., a single Gaussian, to regularize the latent variables. This strategy, however, is ineffective when the user is simultaneously interested in different kinds of items, i.e., the user’s preference may be highly diverse. In this paper, thus, we propose a Deep Global and Local Generative Model for recommendation to consider both local and global structure among users (DGLGM) under the Wasserstein auto-encoder framework. Besides keeping the global structure like the existing model, DGLGM adopts a non-parametric Mixture Gaussian distribution with several components to capture the diversity of the users’ preferences. Each component is corresponding to one local structure and its optimal size can be determined via the automatic relevance determination technique. These two parts can be seamlessly integrated and enhance each other. The proposed DGLGM can be efficiently inferred by minimizing its penalized upper bound with the aid of local variational optimization technique. Meanwhile, we theoretically analyze its generalization error bounds to guarantee its performance in sparse feedback data with diversity. By comparing with the state-of-the-art methods, the experimental results demonstrate that DGLGM consistently benefits the recommendation system in top-N recommendation task.

Interpretability of recommender systems has caused increasing attention due to its promotion of the effectiveness and persuasiveness of recommendation decision, and thus user satisfaction. Most existing methods, such as Matrix Factorization (MF), tend to be black-box machine learning models that lack interpretability and do not provide a straightforward explanation for their outputs. In this paper, we focus on probabilistic factorization model and further assume the absence of any auxiliary information, such as item content or user review. We propose an influence mechanism to evaluate the importance of the users' historical data, so that the most related users and items can be selected to explain each predicted rating. The proposed method is thus called Influencebased Interpretable Recommendation model (In2Rec). To further enhance the recommendation accuracy, we address the important issue of missing not at random, i.e., missing ratings are not independent from the observed and other unobserved ratings, because users tend to only interact what they like. In2Rec models the generative process for both observed and missing data, and integrates the influence mechanism in a Bayesian graphical model. A learning algorithm capitalizing on iterated condition modes is proposed to tackle the non-convex optimization problem pertaining to maximum a posteriori estimation for In2Rec. A series of experiments on four real-world datasets (Movielens 10M, Netflix, Epinions, and Yelp) have been conducted. By comparing with the state-of-the-art recommendation methods, the experimental results have shown that In2Rec can consistently benefit the recommendation system in both rating prediction and ranking estimation tasks, and friendly interpret the recommendation results with the aid of the proposed influence mechanism.

Recommender systems offer critical services in the age of mass information. Personalized ranking has been attractive both for content providers and customers due to its ability of creating a user-specific ranking on the item set. Although the powerful factor-analysis methods including latent factor models and deep neural network models have achieved promising results, they still suffer from the challenging issues, such as sparsity of recommendation data, uncertainty of optimization, and etc. To enhance the accuracy and generalization of recommender system, in this paper, we propose a deep generative ranking (DGR) model under the Wasserstein autoencoder framework. Specifically, DGR simultaneously generates the pointwise implicit feedback data (via a Beta-Bernoulli distribution) and creates the pairwise ranking list by sufficient exploiting both interacted and non-interacted items for each user. DGR can be efficiently inferred by minimizing its penalized evidence lower bound. Meanwhile, we theoretically analyze the generalization error bounds of DGR model to guarantee its performance in extremely sparse feedback data. A series of experiments on four large-scale datasets (Movielens (20M), Netflix, Epinions and Yelp in movie, product and business domains) have been conducted. By comparing with the state-of-the-art methods, the experimental results demonstrate that DGR consistently benefit the recommendation system in ranking estimation task, especially for the near-cold-start-users (with less than five interacted items).

According to the sparseness of rating information, the quality of recommender systems has been greatly restricted. In order to solve this problem, much auxiliary information has been used, such as social networks, review information, and item description. Convolutional neural networks (CNNs) have been widely employed by recommender systems, it greatly improved the rating prediction's accuracy especially when combined with traditional recommendation methods. However, a large amount of research focuses on the consistency between the rating-based latent factor and review-based latent factor. But in fact, these two parts are completely different. In this article, the authors propose a model named collaboration matrix factorization (CMF) that combines a projection method with a convolutional matrix factorization (ConvMF) to extract the collaboration between rating-based latent factors and review-based latent factors that comes from the results of the CNN process. Extensive experiments on three real-world datasets show that the projection method achieves significant improvements over the existing baseline.

Low-rank matrix approximation (LRMA) has attracted more and more attention in the community of recommendation. Even though LRMA-based recommendation methods (including Global LRMA and Local LRMA) obtain promising results, they suffer from the complicated structure of the large-scale and sparse rating matrix, especially when the underlying system includes a large set of items with various types and a huge amount of users with diverse interests. Thus, they have to predefine the important parameters, such as the rank of the rating matrix and the number of submatrices. Moreover, most existing Local LRMA methods are usually designed in a two-phase separated framework and do not consider the missing mechanisms of rating matrix. In this article, a non-parametric unified Bayesian graphical model is proposed for A daptive Lo cal low-rank M atrix A pproximation ( ALoMA ). ALoMA has ability to simultaneously identify rating submatrices, determine the optimal rank for each submatrix, and learn the submatrix-specific user/item latent factors. Meanwhile, the missing mechanism is adopted to characterize the whole rating matrix. These four parts are seamlessly integrated and enhance each other in a unified framework. Specifically, the user-item rating matrix is adaptively divided into proper number of submatrices in ALoMA by exploiting the Chinese Restaurant Process. For each submatrix, by considering both global/local structure information and missing mechanisms, the latent user/item factors are identified in an optimal latent space by adopting automatic relevance determination technique. We theoretically analyze the model’s generalization error bounds and give an approximation guarantee. Furthermore, an efficient Gibbs sampling-based algorithm is designed to infer the proposed model. A series of experiments have been conducted on six real-world datasets ( Epinions , Douban , Dianping , Yelp , Movielens (10M), and Netflix ). The results demonstrate that ALoMA outperforms the state-of-the-art LRMA-based methods and can easily provide interpretable recommendation results.

Recommender system is a popular tool to accurately and actively provide users with potentially interesting information. For capturing the users' preferences and approximating the missing data, matrix completion and approximation are widely adopted. Except for the typical low-rank factorization-based methods, the additive co-clustering approach (ACCAMS) is recently proposed to succinctly approximate large-scale rating matrix. Although ACCAMS efficiently produces effective recommendation result, it still suffers from the cold-start problem. To address this issue, we propose a Social Influence Additive Co-Clustering method (SIACC) by making use of user-item rating data and user-user social relations.