Real-world scientific applications frequently encounter incomplete observational data due to sensor limitations, geographic constraints, or measurement costs. Although neural operators significantly advanced PDE solving in terms of computational efficiency and accuracy, their underlying assumption of fully-observed spatial inputs severely restricts applicability in real-world application. We introduce the first systematic framework for learning neural operators from partial observation. We identify and formalize two fundamental obstacles: (i) the supervision gap in unobserved regions that prevents effective learning of physical correlations, and (ii) the dynamic spatial mismatch between incomplete inputs and complete solution fields. Specifically, our proposed LANO (Latent Autoregressive Neural Operator) introduces two novel components designed explicitly to address the core difficulties of partial observations: (i) a mask-to-predict training strategy that creates artificial supervision by strategically masking observed regions, and (ii) a Physics-Aware Latent Propagator that reconstructs solutions through boundary-first autoregressive generation in latent space. Additionally, we develop POBench-PDE, a dedicated and comprehensive benchmark designed specifically for evaluating neural operators under partial observation conditions across three PDE-governed tasks. LANO achieves state-of-the-art performance with relative error reductions ranging from eighteen to sixty-nine percent across all benchmarks under patch-wise missingness with missing rates below fifty percent, including real-world climate prediction. Our approach effectively addresses practical scenarios with missing rates of up to seventy-five percent, to some extent bridging the existing gap between idealized research settings and the complexities of real-world scientific computing.

Meta-learning for Bayesian optimization accelerates optimization by leveraging knowledge from previous tasks, but existing methods optimize for average performance and fail on challenging outlier tasks critical in practice. These limitations become particularly severe when target tasks exhibit distribution shifts or when optimization budgets are limited in real-world applications. We introduce MetaGameBO, a hierarchical game-theoretic framework that formulates meta-learning as robust optimization through CVaR-based task selection and diversity-aware sample learning. Our approach incorporates uncertainty-aware adaptation via probabilistic embeddings and Thompson sampling for robust generalization to out-of-distribution targets. We establish theoretical guarantees including convergence to game-theoretic equilibria and improved sample complexity, and demonstrate substantial improvements with 95.7% reduction in average loss and 88.6% lower tail risk compared to state-of-the-art methods on challenging tasks and distribution shifts.

Realistic choreography demands simultaneous attention to rhythm and motivation. Prevailing automated dance generation methods mainly depend on musical input, overlooking the motivations that drive meaningful dance creation. Inspired by the motivation choreography, we aim to articulate dance motivations through textual guidance. However, the absence of high-quality datasets concurrently containing music, textual descriptions, and motion data presents a challenge in achieving accurate fine-grained textual control. To address this limitation, we present MotivDance, a novel framework integrating fine-grained textual guidance with music to synthesize semantically coherent dance sequences. Our approach first synthesizes text-guided key poses as motivations. We then introduce an Adaptive Keyframe Locator that dynamically positions these motivations within the musical context through beat-aware synchronization and cross-modal latent space alignment. Finally, a Transformer-based U-Net diffusion model performs the motion in-betweening while preserving motivational integrity. Extensive qualitative and quantitative experiments demonstrate that MotivDance effectively integrates music with fine-grained text control to generate high-fidelity dance motions.

Speech-driven 3D motion generation has garnered increasing research attention. However, it faces significant challenges in achieving style controllability, primarily due to the scarcity of motion style annotations. To address this, we propose a novel diffusion-based framework for co-speech holistic motion generation that enables example-based style control from videos. Our approach integrates hierarchical speech encoding with rhythm-aware denoising to produce natural and synchronized gestures and expressions. For effective style guidance, we introduce a contrastive style encoder that captures discriminative style representations from reference clips without explicit labeling, enabling generalization to motion styles unseen during training. Furthermore, we design a neural mapper that aligns 2D and 3D gesture features in a shared embedding space, facilitating direct style extraction from in-the-wild videos and seamless transfer to 3D motion. Extensive experiments and user studies show that our proposed approach achieves state-of-the-art performance in both qualitative and quantitative evaluations, offering a flexible solution for controllable motion generation.

We propose UAD, an end-to-end framework with Unsupervised pretext task for vision-based Autonomous Driving, achieving the best open-loop evaluation performance in nuScenes, meanwhile showing robust closed-loop driving quality in CARLA. Our motivation stems from the observation that current end-to-end autonomous driving (E2EAD) models still mimic the modular architecture in typical driving stacks, with carefully designed supervised perception and prediction subtasks to provide environment information for oriented planning. Although achieving groundbreaking progress, such design has certain drawbacks: 1) preceding subtasks require massive high-quality 3D annotations as supervision, posing a significant impediment to scaling the training data; and 2) each submodule entails substantial computation overhead in both training and inference. To this end, we propose UAD, an E2EAD framework with an unsupervised proxy to address all these issues. Firstly, we design a novel Angular Perception Pretext to eliminate the annotation requirement. The pretext perceives the driving scene by predicting the angular-wise spatial objectness and temporal dynamics, without manual annotation. Secondly, a self-supervised training strategy, which learns the consistency of the predicted trajectories under different augment views, is proposed to enhance the planning robustness in steering scenarios. Our UAD achieves 38.7% relative improvements over UniAD on the average collision rate of nuScenes open-loop evaluation and obtains the route completion score of 98.5% in closed-loop evaluation of CARLA's Town05 Long benchmark, which outperforms the recent work VADv2. Moreover, the proposed method consumes only 44.3% training resources of UniAD and runs 3.4× faster in inference when employing the same backbone network. Our innovative design not only for the first time demonstrates unarguable performance advantages over supervised counterparts, but also enjoys unprecedented efficiency in data, training, and inference.

Children with Autism Spectrum Disorder (ASD) often misunderstand social situations and struggle to participate in daily routines. Social Stories™ are traditionally crafted by psychology experts under strict constraints to address these challenges but are costly and limited in diversity. As Large Language Models (LLMs) advance, there's an opportunity to develop more automated, affordable, and accessible methods to generate Social Stories in real-time with broad coverage. However, adapting LLMs to meet the unique and strict constraints of Social Stories is a challenging issue. To this end, we propose SS-GEN, a Social Story GENeration framework with LLMs. Firstly, we develop a constraint-driven sophisticated strategy named StarSow to hierarchically prompt LLMs to generate Social Stories at scale, followed by rigorous human filtering to build a high-quality dataset. Additionally, we introduce quality assessment criteria to evaluate the effectiveness of these generated stories. Considering that powerful closed-source large models require very complex instructions and expensive API fees, we finally fine-tune smaller language models with our curated high-quality dataset, achieving comparable results at lower costs and with simpler instruction and deployment. This work marks a significant step in leveraging AI to personalize Social Stories cost-effectively for autistic children at scale, which we hope can encourage future research on special groups.

Continual fine-tuning of Large Language Models (LLMs) is hampered by the trade-off between efficiency and expressiveness.Low-Rank Adaptation (LoRA) offers efficiency but constrains the model's ability to learn new tasks and transfer knowledge due to its lowrank nature and reliance on explicit parameter constraints.We propose GORP (Gradient LOw Rank Projection) for Continual Learning, a novel training strategy that overcomes these limitations by synergistically combining full and low-rank parameters and jointly updating within a unified low-rank gradient subspace.GORP expands the optimization space while preserving efficiency and mitigating catastrophic forgetting.Extensive experiments on continual learning benchmarks demonstrate GORP's superior performance compared to existing state-of-the-art approaches.

Aiming to achieve class-agnostic perception in visual object tracking, current trackers commonly formulate tracking as a one-shot detection problem with the template-matching architecture. Despite the success, severe environmental variations in long-term tracking raise challenges to generalizing the tracker in novel situations. Temporal trackers try to fix it by preserving the time-validity of target information with historical predictions, e.g., updating the template. However, solely transmitting the previous observations instead of learning from them leads to an inferior capability of understanding the tracking scenario from past experience, which is critical for the generalization in new frames. To address this issue, we reformulate temporal learning in visual tracking as a History-To-Future process and propose a novel tracking framework DreamTrack. Our DreamTrack learns the temporal dynamics from past observations to dream the future variations of the environment, which boosts the generalization with the extended future information from history. Considering the uncertainty of future variation, multimodal prediction is designed to infer the target trajectory of each possible future situation. The experiments demonstrate that our DreamTrack achieves leading performance with real-time inference speed. In particular, DreamTrack obtains SUC scores of 76.6%/87.9% on LaSOT/TrackingNet, surpassing all recent SOTA trackers.

We investigate the generalization boundaries of current Large Multimodal Models (LMMs) under out-of-distribution scenarios and domain-specific tasks. We evaluate their zero-shot generalization across synthetic images, real-world distributional shifts, and specialized datasets like medical and molecular imagery. Empirical results indicate that LMMs struggle with generalization beyond common training domains, limiting their direct application without adaptation. To understand the cause of unreliable performance, we analyze three hypotheses: semantic misinterpretation, visual feature extraction insufficiency, and mapping deficiency. Results identify mapping deficiency as the primary hurdle. To address this problem, we show that in-context learning (ICL) can significantly enhance LMMs’ generalization. We further explore the robustness of ICL under distribution shifts and show its vulnerability to domain shifts, label shifts, and spurious correlation shifts between in-context examples and test data, opening new avenues for overcoming generalization barriers.

Yi Feng, Jiaqi Wang, Wenxuan Zhang, Zhuang Chen, Shen Yutong, Xiyao Xiao, Minlie Huang, Liping Jing, Jian Yu. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025.

While the Segment Anything Model (SAM) transforms interactive segmentation with zero-shot abilities, its inherent vulnerabilities present a single-point risk, potentially leading to the failure of numerous downstream applications. Proactively evaluating these transferable vulnerabilities is thus imperative. Prior adversarial attacks on SAM often present limited transferability due to insufficient exploration of common weakness across domains. To address this, we propose Vertex-Refining Simplicial Complex Attack (VeSCA), a novel method that leverages only the encoder of SAM for generating transferable adversarial examples. Specifically, it achieves this by explicitly characterizing the shared vulnerable regions between SAM and downstream models through a parametric simplicial complex. Our goal is to identify such complexes within adversarially potent regions by iterative vertex-wise refinement. A lightweight domain re-adaptation strategy is introduced to bridge domain divergence using minimal reference data during the initialization of simplicial complex. Ultimately, VeSCA generates consistently transferable adversarial examples through random simplicial complex sampling. Extensive experiments demonstrate that VeSCA achieves performance improved by 12.7% compared to state-of-the-art methods across three downstream model categories across five domain-specific datasets. Our findings further highlight the downstream model risks posed by SAM's vulnerabilities and emphasize the urgency of developing more robust foundation models.

Neural processes (NPs) are a promising paradigm to enable skill transfer learning across tasks with the aid of the distribution of functions. The previous NPs employ the empirical risk minimization principle in optimization. However, the fast adaption ability to different tasks can vary widely, and the worst fast adaptation can be catastrophic in risk-sensitive tasks. To achieve robust neural processes modeling, we consider the problem of training models in a risk-averse manner, which can control the worst fast adaption cases at a certain probabilistic level. By transferring the risk minimization problem to a two-level finite sum minimax optimization problem, we can easily solve it via a double-looped stochastic mirror prox algorithm with a task-aware variance reduction mechanism via sampling samples across all tasks. The mirror prox technique ensures better handling of complex constraint sets and non-Euclidean geometries, making the optimization adaptable to various tasks. The final solution, by aggregating prox points with the adaptive learning rates, enables a stable and high-quality output. The proposed learning strategy can work with various NPs flexibly and achieves less biased approximation with a theoretical guarantee. To illustrate the superiority of the proposed model, we perform experiments on both synthetic and real-world data, and the results demonstrate that our approach not only helps to achieve more accurate performance but also improves model robustness.

Continuous-time dynamic modeling of irregular time series is crucial for capturing the continuous evolution and complex correlations within the data. Traditional methods, such as Recurrent Neural Network (RNN)-based models, often require discretization of the time series, which may lead to information loss or insufficient capture of dynamic relationships between data points. Although Neural Ordinary Differential Equations (Neural ODEs) and their variants have shown some potential in handling irregular time series, they often struggle to effectively address the stochasticity in the data and capture complex dynamic relationships within the sequences. To address these issues, we propose SDEFormer, a continuous-time dynamic Transformer model based on Neural Stochastic Differential Equations (Neural SDEs). SDEFormer combines the continuous-time modeling capabilities of Neural SDEs with the sequence modeling strengths of Transformers, enabling it to capture both deterministic and stochastic components in complex continuous-time dynamics. We mathematically describe the expressive power of SDEFormer and address the stability issues inherent in SDEs by designing three stable SDE-based variants. Experiments on multiple irregular time series classification and event prediction tasks demonstrate that SDEFormer exhibits stronger robustness when dealing with sparse and irregular data, outperforming traditional RNNs, ODE methods, and some Transformer variants in terms of prediction accuracy and stability. By integrating continuous-time modeling with attention mechanisms, SDEFormer effectively captures complex temporal dependencies and system behaviors in irregular time series, showcasing its powerful capabilities in irregular time series analysis.

Existing work in physical-informed machine learning (PIML) has shown that data-driven learning of solution operators can provide a fast approximate alternative to classical numerical ordinary/partial differential equations (ODEs/PDEs) solvers. Of these, Neural Operators (NOs) have emerged as particularly promising. However, a key challenge in the field of NOs lies in developing methods that can effectively handle out-of-distribution (OOD) forecasting problems. Such problems involve the ability to adaptively learn from observations of the same dynamical system governed by ODEs/PDEs, where the underlying parameters are unknown and vary across instances. These tasks further require precise predictions even when faced with initial conditions and PDEs/ODEs parameters outside the training distribution. In this study, we consider the problem of training models in a risk-reverse manner. We introduce a risk-aware framework aimed at enhancing the OOD robustness of NOs by stochastically optimizing the conditional value-at-risk (CVAR) of a loss distribution. Through experiments on different distinct OOD tasks, our approach demonstrates a significant performance improvement over existing advanced NOs.

Information diffusion prediction is fundamental for forecasting user participation in information sharing on social networks, such as retweets on Twitter. Existing methods typically extract user relationships from social networks and historical interactions, while further capturing contextual information within the specific diffusion process. However, these methods have several limitations: (1) They often utilize sequential diffusion process for prediction and simplify differentiated influences among participants; (2) They capture user relationships on the entire graph for all users, in which most information is not necessary for a specific diffusion process and is too inefficient for real-world large-scale networks. To tackle these limitations, we propose a novel and scalable model SILN, for sphere-based information diffusion prediction on large social networks. Specifically, SILN features three components. First, we integrate two kinds of sphere effects in terms of structural and temporal views, which learn an enhanced cascade representation. Second, SILN designs an efficient learning scheme based on the cascade-specific subgraph, which significantly reduces the entire graph computation to smaller subgraphs. Third, to facilitate subgraph extraction, we develop an optimized graph storage technique to allow constant-time neighbor access and reduce the storage cost by about 30% in practice. Extensive experiments on six real-world datasets validate that SILN consistently outperforms seven state-of-the-art competitors in prediction performance while exhibiting exceptional time and space efficiency on million-node social networks.

Large language models (LLMs) have demonstrated impressive capabilities, but their enormous size poses significant challenges for deployment in real-world applications. To address this issue, researchers have sought to apply network pruning techniques to LLMs. A critical challenge in pruning is the allocation of sparsity for each layer. Recent sparsity allocation methods are often based on heuristics or search that can easily lead to suboptimal performance. In this paper, we conducted an extensive investigation into various LLMs and revealed three significant discoveries: (1) the Layerwise Pruning Sensitivity (LPS) of LLMs is highly non-uniform, (2) the choice of pruning metric affects LPS, and (3) the performance of a sparse model is related to the uniformity of its layerwise redundancy level. Based on these discoveries, we propose that the layerwise sparsity of LLMs should adhere to three principles: non-uniformity, pruning metric dependency, and uniform layerwise redundancy level in the pruned model. To this end, we proposed Maximum Redundancy Pruning (MRP), an iterative pruning algorithm that prunes in the most redundant layers (i.e., those with the highest non-outlier ratio) at each iteration. The achieved layerwise sparsity aligns with the outlined principles. We conducted extensive experiments on publicly available LLMs, including LLaMA2 and OPT, on various benchmarks. The experimental results validate the effectiveness of MRP, demonstrating its superiority over previous methods.

Neural Processes (NPs) combine the adaptability of neural networks with the efficiency of meta-learning, offering a powerful framework for modeling stochastic processes. However, existing methods focus on empirical performance while lacking a rigorous theoretical understanding of generalization. To address this, we propose an information-theoretic framework to analyze the generalization bounds of NPs, introducing dynamical stability regularization to minimize sharpness and improve optimization dynamics. Additionally, we show how noise-injected parameter updates complement this regularization. The proposed approach, applicable to a wide range of NP models, is validated through experiments on classic benchmarks, including 1D regression, image completion, Bayesian optimization, and contextual bandits. The results demonstrate tighter generalization bounds and superior predictive performance, establishing a principled foundation for advancing generalizable NP models.

Strong adversarial examples are crucial for evaluating and enhancing the robustness of deep neural networks. However, the performance of popular attacks is usually sensitive, for instance, to minor image transformations, stemming from limited information ─ typically only one input example, a handful of white-box source models, and undefined defense strategies. Hence, the crafted adversarial examples are prone to overfit the source model, which hampers their transferability to unknown architectures. In this paper, we propose an approach named Multiple Asymptotically Normal Distribution Attacks (MultiANDA) which explicitly characterize adversarial perturbations from a learned distribution. Specifically, we approximate the posterior distribution over the perturbations by taking advantage of the asymptotic normality property of stochastic gradient ascent (SGA), then employ the deep ensemble strategy as an effective proxy for Bayesian marginalization in this process, aiming to estimate a mixture of Gaussians that facilitates a more thorough exploration of the potential optimization space. The approximated posterior essentially describes the stationary distribution of SGA iterations, which captures the geometric information around the local optimum. Thus, MultiANDA allows drawing an unlimited number of adversarial perturbations for each input and reliably maintains the transferability. Our proposed method outperforms ten state-of-the-art black-box attacks on deep learning models with or without defenses through extensive experiments on seven normally trained and seven defense models.

Vision-centric autonomous driving has recently raised wide attention due to its lower cost. Pretraining is essential for extracting a universal representation. However, current vision-centric pretraining typically relies on either 2D or 3D pre-text tasks, overlooking the temporal characteristics of autonomous driving as a 4D scene understanding task. In this paper, we address this challenge by introducing a world model-based autonomous driving 4D representation learning framework, dubbed DriveWorld, which is capable of pretraining from multi-camera driving videos in a spatiotemporal fashion. Specifically, we propose a Memory State-Space Model for spatiotemporal modelling, which consists of a Dynamic Memory Bank module for learning temporal-aware latent dynamics to predict future changes and a Static Scene Propagation module for learning spatial-aware latent statics to offer comprehensive scene contexts. We additionally introduce a Task Prompt to decouple task-aware features for various downstream tasks. The experiments demonstrate that DriveWorld delivers promising results on various autonomous driving tasks. When pretrained with the OpenScene dataset, DriveWorld achieves a 7.5% increase in mAP for 3D object detection, a 3.0% increase in IoU for online mapping, a 5.0% increase in AMOTA for multi-object tracking, a 0.1m decrease in minADE for motionforecasting, a 3.0% increase in IoU for occupancy prediction, and a 0.34m reduction in average L2 error for planning.

In extreme multi-label text classification (XMC), labels usually follow a long-tailed distribution, where most labels only contain a small number of documents and limit the performance of XMC. Data augmentation (DA) is a simple but effective strategy to solve such low-resource problems. In this paper, we propose a prompt-based DA method called XDA, which is specifically designed for XMC. First, we employ a soft prompt during the fine-tuning process of the T5 model for label-conditional DA, thereby enabling T5 to augment samples while preserving label-compatibility. Subsequently, XDA performs sample filtering on the augmented samples through the diversity of text and the consistency of labels, which enhances the quality of the DA. In contrast to traditional sample-level DA, we propose a pair-level DA method by masking the augmented sample-label pairs of head-labels during training, effectively mitigating the long-tailed problem. Comprehensive experiments on benchmark datasets have shown that the proposed XDA outperforms the state-of-the-art counterparts.

Most works of interpretable neural networks strive for learning the semantics concepts merely from single modal information such as images. However, humans usually learn semantic concepts from multiple modalities and the semantics is encoded by the brain from fused multi-modal information. Inspired by cognitive science and vision-language learning, we propose a Prototype-Concept Alignment Network (ProCoNet) for learning visual prototypes under the guidance of textual concepts. In the ProCoNet, we have designed a visual encoder to decompose the input image into regional features of prototypes, while also developing a prompt generation strategy that incorporates in-context learning to prompt large language models to generate textual concepts. To align visual prototypes with textual concepts, we leverage the multimodal space provided by the pre-trained CLIP as a bridge. Specifically, the regional features from the vision space and the cropped regions of prototypes encoded by CLIP reside on different but semantically highly correlated manifolds, i.e. follow a multi-manifold distribution. We transform the multi-manifold distribution alignment problem into optimizing the projection matrix by Cayley transform on the Stiefel manifold. Through the learned projection matrix, visual prototypes can be projected into the multimodal space to align with semantically similar textual concept features encoded by CLIP. We conducted two case studies on the CUB-200-2011 and Oxford Flower dataset. Our experiments show that the ProCoNet provides higher accuracy and better interpretability compared to the single-modality interpretable model. Furthermore, ProCoNet offers a level of interpretability not previously available in other interpretable methods.

Abstract We propose a unified object-aware temporal learning framework for multi-view 3D detection and tracking tasks. Having observed that the efficacy of the temporal fusion strategy in recent multi-view perception methods may be weakened by distractors and background clutters in historical frames, we propose a cyclic learning mechanism to improve the robustness of multi-view representation learning. The essence is constructing a backward bridge to propagate information from model predictions (e.g., object locations and sizes) to image and BEV features, which forms a circle with regular inference. After backward refinement, the responses of target-irrelevant regions in historical frames would be suppressed, decreasing the risk of polluting future frames and improving the object awareness ability of temporal fusion. We further tailor an object-aware association strategy for tracking based on the cyclic learning model. The cyclic learning model not only provides refined features, but also delivers finer clues (e.g., scale level) for tracklet association. The proposed cycle learning method and association module together contribute a novel and unified multi-task framework. Experiments on nuScenes show that the proposed model achieves consistent performance gains over baselines of different designs (i.e., dense query-based BEVFormer, sparse query-based SparseBEV and LSS-based BEVDet4D) on both detection and tracking evaluation. Codes and models will be released.

Multimodal vision-language (VL) learning has noticeably pushed the tendency toward generic intelligence owing to emerging large foundation models. However, tracking, as a fundamental vision problem, surprisingly enjoys less bonus from recent flourishing VL learning. We argue that the reasons are two-fold: the lack of large-scale vision-language annotated videos and ineffective vision-language interaction learning of current works. These nuisances motivate us to design more effective vision-language representation for tracking, meanwhile constructing a large database with language annotation for model learning. Particularly, in this paper, we first propose a general attribute annotation strategy to decorate videos in six popular tracking benchmarks, which contributes a large-scale vision-language tracking database with more than 23,000 videos. We then introduce a novel framework to improve tracking by learning a unified-adaptive VL representation, where the cores are the proposed asymmetric architecture search and modality mixer (ModaMixer). To further improve VL representation, we introduce a contrastive loss to align different modalities. To thoroughly evidence the effectiveness of our method, we integrate the proposed framework on three tracking methods with different designs, i.e., the CNN-based SiamCAR (Guo et al. 2020), the Transformer-based OSTrack (Ye et al. 2022), and the hybrid structure TransT (Chen et al. 2021). The experiments demonstrate that our framework can significantly improve all baselines on six benchmarks. Besides empirical results, we theoretically analyze our approach to show its rationality. By revealing the potential of VL representation, we expect the community to divert more attention to VL tracking and hope to open more possibilities for future tracking with diversified multimodal messages.

When humans explain their reasoning, such as their classification decisions, they often break down an image into parts and highlight the evidence from those parts to support the concepts they have in mind. Drawing inspiration from this cognitive process, several self-explaining models have been proposed to explain predictions by part-level concepts. However, these models can be limited by their structure and difficulty in determining the effect of individual parts on the output category. To address these challenges, we introduce a self-explaining architecture that uses a plug-in transparent embedding space (TesNet) to connect high-level input patches (e.g. feature maps or tokens) with output categories. The transparent embedding space is spanned by basis concepts and constructed on the Grassmann manifold. The basis concepts are enforced to be category-aware, and within-category concepts are orthogonal to each other, ensuring the embedding space is disentangled. To reduce concept redundancy and restore the concept space structure, we introduce two concept pruning methods and a new re-training strategy to build a slimming transparent embedding space. We verify the scalability of TesNet through experiments on deep networks such as VGG, ResNet, DenseNet, and Vision Transformer. Additionally, we design several metrics for self-explaining models to quantify interpretability and compare them with state-of-the-art self-explaining methods. Our experiments demonstrate that TesNet is much more effective for classification tasks, providing better interpretability on predictions and improving final accuracy.

As an important branch of unsupervised learning methods, clustering makes a wide contribution in the area of data mining. It is well known that capturing the group-discriminative properties of each sample for clustering is crucial. Among them, deep clustering delivers promising results due to the strong representational power of neural networks. However, most of them adopt sample-level learning strategies, and the standalone data point barely captures its holistic cluster’s context and may undergo sub-optimal cluster assignment. To tackle this issue, we propose a Structure-driven Representation Learning (SRL) method by introducing latent structure information into the representation learning process at both the local and global levels. Specifically, a local-structure-driven sample representation strategy is proposed to approximate the estimation of data distribution, which models the neighborhood distribution of samples with potential structure information and exploits statistical dependencies between them to improve cluster consistency. A global-structure-driven cluster representation strategy is designed, where the context of each cluster is sufficiently encoded according to its samples (exemplar-theory) and corresponding prototype (prototype-theory). In this case, each cluster can only be related to its most similar samples, and different clusters are separated as much as possible. These two models are seamlessly combined into a joint optimization problem, which can be efficiently solved. Experiments on six widely-used datasets demonstrate the superiority of SRL over state-of-the-art clustering methods.

User preferences behind users' decision-making processes are highly diverse and may range from lower-level concepts with more specific intentions and higher-level concepts with more general intentions. In this case, user preferences tend to be expressed hierarchically. However, learning such intentions with different levels from user behaviors is challenging, and remains largely neglected by the existing literature. Meanwhile, user behavior data tends to be sparse because of the limited user response and the vast combinations of users and items, which results in cold-start problems with unclear user intentions. In this paper, we propose a mixture intention neural stochastic process (MINSP), a new view of the stochastic processes family using a general meta-learning mechanism and mixture strategy for robust recommendation with hierarchical preferences modeling. By considering the recommendation process for each user as a stochastic process, MINSP defines distributions over functions and is capable of rapid adaptation to different users. To capture the user's intention on different levels, an iterative additive algorithm is proposed that minimizes the approximation error by backfitting the residuals of previous approximations. In this case, the induced tree intention hierarchies serve as an aggregated structured representation of the whole preference, summarizing the gist for convenient navigation and better generalization. Furthermore, we theoretically analyze the generalization error bound of the proposed MINSP to guarantee the model performance. Empirical results show that our approach can achieve substantial improvement over the state-of-the-art baselines in terms of recommendation performance, and obtain an interpretable hierarchical intention structure.

Multi-label text classification (MLTC) involves tagging a document with its most relevant subset of labels from a label set. In real applications, labels usually follow a long-tailed distribution, where most labels (called as tail-label) only contain a small number of documents and limit the performance of MLTC. To facilitate this low-resource problem, researchers introduced a simple but effective strategy, data augmentation (DA). However, most existing DA approaches struggle in multi-label settings. The main reason is that the augmented documents for one label may inevitably influence the other co-occurring labels and further exaggerate the long-tailed problem. To mitigate this issue, we propose a new pair-level augmentation framework for MLTC, called Label-Specific Feature Augmentation (LSFA), which merely augments positive feature-label pairs for the tail-labels. LSFA contains two main parts. The first is for label-specific document representation learning in the high-level latent space, the second is for augmenting tail-label features in latent space by transferring the documents second-order statistics (intra-class semantic variations) from head labels to tail labels. At last, we design a new loss function for adjusting classifiers based on augmented datasets. The whole learning procedure can be effectively trained. Comprehensive experiments on benchmark datasets have shown that the proposed LSFA outperforms the state-of-the-art counterparts.

Keyphrase extraction aims to provide readers with high-level information about the central ideas or important topics described in a given source text. Recent advances in embedding-based models have made remarkable progress on unsupervised keyphrase extraction, demonstrated through improved quality metrics such as F1-score. However, the diversity in the keyphrase extraction task needs to be addressed. In this paper, we focus on diverse keyphrase extraction, which entails extracting keyphrases that cover different central information or essential topics in the document. To achieve this goal, we propose a re-ranking-based approach that employs determinantal point processes utilizing BERT as kernels, which we call DiversityRank. Specifically, DiversityRank jointly considers phrase-document relevance and cross-phrase similarities to select candidate keyphrases that are document-relevant and diverse. Results demonstrate that our re-ranking strategy outperforms the state-of-the-art unsupervised keyphrase extraction baselines on three benchmark datasets.

=User cold-start recommendation is a well-known challenge in current recommender systems. The cause is that the number of user interactions is too few to accurately estimate user preferences. Furthermore, the uncertainty of user interactions intensifies along with the number of user interactions decreasing. Although existing meta-learning based models with globally sharing knowledge show good performance in most cold-start scenarios, the ability of handling challenges on intention importance and prediction uncertainty is missing: (1) Intra-user uncertainty. When estimating user preferences (reflected in the user's latent representation), each of user interactions is independently considered in the form of user-item pair, which cannot capture the correlation between user interactions, as well as considering the global intent under user interactions. (2) Inter-user importance. During the model training, all users are treated as equally important, which cannot distinguish the contribution of users in the model training process. Assigning the same weight to all users may lead to users with high uncertainty incorrectly guiding the model learning in the early stage of training. To tackle the above challenges, in this paper, we focus on modeling user preference as a weighted distribution over functions (WDoF) for user cold-start recommendation, which not only models the intra-user uncertainty through neural processes with Multinomial likelihood but also considers the importance of different users with curriculum learning during the model training process. Furthermore, we provide a theoretical explanation that why the proposed model performs better than regular neural processes based recommendation methods. Experiments on four real-world datasets demonstrate the effectiveness of the proposed model over several state-of-the-art cold-start recommendation methods.

Given the exponential growth in the number of documents on the web in recent years, there is an increasing demand for accurate models to extract keyphrases from such documents.Keyphrase extraction is the task of automatically identifying representative keyphrases from the source document.Typically, candidate keyphrases exhibit latent hierarchical structures embedded with intricate syntactic and semantic information.Moreover, the relationships between candidate keyphrases and the document also form hierarchical structures.Therefore, it is essential to consider these latent hierarchical structures when extracting keyphrases.However, many recent unsupervised keyphrase extraction models overlook this aspect, resulting in incorrect keyphrase extraction.In this paper, we address this issue by proposing a new hyperbolic ranking model (HyperRank).HyperRank is designed to jointly model global and local context information for estimating the importance of each candidate keyphrase within the hyperbolic space, enabling accurate keyphrase extraction.Experimental results demonstrate that HyperRank significantly outperforms recent state-of-the-art baselines.

Over-generation errors occur when a keyphrase extraction model correctly determines a candidate keyphrase as a keyphrase because it contains a word that frequently appears in the document but at the same time erroneously outputs other candidates as keyphrases because they contain the same word.To mitigate this issue, we propose a new heterogeneous centrality detection approach (CentralityRank), which extracts keyphrases by simultaneously identifying both implicit and explicit centrality within a heterogeneous graph as the importance score of each candidate.More specifically, Centrali-tyRank detects centrality by taking full advantage of the content within the input document to construct graphs that encompass semantic nodes of varying granularity levels, not limited to just phrases.These additional nodes act as intermediaries between candidate keyphrases, enhancing inter-phrase relevance.Furthermore, we introduce a novel adaptive boundary-aware regularization that can leverage the position information of candidate keyphrases, thus influencing the importance of candidate keyphrases.Extensive experimental results demonstrate the superiority of CentralityRank over recent stateof-the-art unsupervised keyphrase extraction baselines on three benchmark datasets.

Information Bottlenecks (IBs) learn representations that generalize to unseen data by information compression. However, existing IBs are practically unable to guarantee generalization in real-world scenarios due to the vacuous generalization bound. The recent PAC-Bayes IB uses information complexity instead of information compression to establish a connection with the mutual information generalization bound. However, it requires the computation of expensive second-order curvature, which hinders its practical application. In this paper, we establish the connection between the recognizability of representations and the recent functional conditional mutual information (f-CMI) generalization bound, which is significantly easier to estimate. On this basis we propose a Recognizable Information Bottleneck (RIB) which regularizes the recognizability of representations through a recognizability critic optimized by density ratio matching under the Bregman divergence. Extensive experiments on several commonly used datasets demonstrate the effectiveness of the proposed method in regularizing the model and estimating the generalization gap.

Long-term time series forecasting facilitates early decision-making in a variety of fields, including weather forecasting and disease control. For the majority of data, the only way to predict the future is by discovering the hidden regularities of historical data. However, these regularities are difficult to learn. In the real world, time series patterns are complex, with a mixture of various trends and seasonality, and these sub-patterns are distinct. Thus, it is challenging for a single model to learn the regularities underlying these distinct sub-patterns. Second, these regularities are frequently obscured by noise, which makes it easy to overfit during the learning process. To address these problems, we propose a Bootstrap Decomposition Enhanced Orthogonal-Basis Projection model, or BDEOP: it first decomposes the data into several distinct sub-patterns with a bootstrap decomposition module; subsequently, it uses different modules to capture the hidden regularities and make the predictions based on these sub-patterns; simultaneously, the effect of noise is eliminated by randomly dropping the frequency components in the Fourier space. In terms of accuracy, our experimental investigation of four benchmark datasets from multiple domains demonstrates that the proposed model outperforms state-of-the-art models. Code is available at https://github.com/Nicholas0917/BDEOP-main.

The cold-start recommendation has been one of the most central problems in online platforms where new users or items arrive continuously. Although existing meta-learning based models with globally sharing knowledge show good performance in most cold-start scenarios, the ability to handle challenges on intention heterogeneity and prediction uncertainty is missing, and these two challenges are particularly evident in cold-start scenarios with fewer interaction data. To tackle the above challenges, in this paper, we present an uncertainty-aware Stochastic Meta Process with Doubly Intention learning (DISMP) for the cold-start recommendation, which has promising properties in uncertainty quantification. With the aid of the meta-learning stochastic process, DISMP can store general knowledge by capturing the relevance of different user-item pairs in terms of intentions and concepts, which is capable of rapid adaptation to new users and items. Furthermore, intentions with general and specific levels are extracted by doubly distinguishing the role of latent variables, which is able to capture the dependencies across different types of intentions and concepts. Empirical results show that our approach can achieve substantial improvement over the state-of-the-art baselines on cold-start recommendations with different perspectives.

Unlike traditional statistical models depending on hand-specified priors, neural processes (NPs) have recently emerged as a class of powerful neural statistical models that combine the strengths of neural networks and stochastic processes. NPs can define a flexible class of stochastic processes well suited for highly non-trivial functions by encoding contextual knowledge into the function space. However, noisy context points introduce challenges to the algorithmic stability that small changes in training data may significantly change the models and yield lower generalization performance. In this paper, we provide theoretical guidelines for deriving stable solutions with high generalization by introducing the notion of algorithmic stability into NPs, which can be flexible to work with various NPs and achieves less biased approximation with theoretical guarantees. To illustrate the superiority of the proposed model, we perform experiments on both synthetic and real-world data, and the results demonstrate that our approach not only helps to achieve more accurate performance but also improves model robustness.

Continual learning entails learning a sequence of tasks and balancing their knowledge appropriately. With limited access to old training samples, much of the current work in deep neural networks has focused on overcoming catastrophic forgetting of old tasks in gradient-based optimization. However, the normalization layers provide an exception, as they are updated interdependently by the gradient and statistics of currently observed training samples, which require specialized strategies to mitigate recency bias. In this work, we focus on the most popular Batch Normalization (BN) and provide an in-depth theoretical analysis of its sub-optimality in continual learning. Our analysis demonstrates the dilemma between balance and adaptation of BN statistics for incremental tasks, which potentially affects training stability and generalization. Targeting on these particular challenges, we propose Adaptive Balance of BN (AdaB$^2$N), which incorporates appropriately a Bayesian-based strategy to adapt task-wise contributions and a modified momentum to balance BN statistics, corresponding to the training and testing stages. By implementing BN in a continual learning fashion, our approach achieves significant performance gains across a wide range of benchmarks, particularly for the challenging yet realistic online scenarios (e.g., up to 7.68\%, 6.86\% and 4.26\% on Split CIFAR-10, Split CIFAR-100 and Split Mini-ImageNet, respectively). Our code is available at https://github.com/lvyilin/AdaB2N.

Extractive summarization helps provide a short description or a digest of news or other web texts. It enhances the reading experience of users, especially when they are reading on small displays (e.g., mobile phones). Matching-based methods are recently proposed for the extractive summarization task, which extracts a summary from a global view via a document-summary matching framework. However, these methods only calculate similarities between candidate summaries and the entire document embeddings, insufficiently capturing interactions between different contextual information in the document to accurately estimate the importance of candidates. In this paper, we propose a new hyperbolic interaction model for extractive multi-document summarization (HISum). Specifically, HISum first learns document and candidate summary representations in the same hyperbolic space to capture latent hierarchical structures and then estimates the importance scores of candidates by jointly modeling interactions between each candidate and the document from global and local views. Finally, the importance scores are used to rank and extract the best candidate as the extracted summary. Experimental results on several benchmarks show that HISum outperforms the state-of-the-art extractive baselines1.

Discovering hidden pattern from imbalanced data is a critical issue in various real-world applications. Existing classification methods usually suffer from the limitation of data especially for minority classes, and result in unstable prediction and low performance. In this paper, a deep generative classifier is proposed to mitigate this issue via both model perturbation and data perturbation. Specially, the proposed generative classifier is derived from a deep latent variable model where two variables are involved. One variable is to capture the essential information of the original data, denoted as latent codes, which are represented by a probability distribution rather than a single fixed value. The learnt distribution aims to enforce the uncertainty of model and implement model perturbation, thus, lead to stable predictions. The other variable is a prior to latent codes so that the codes are restricted to lie on components in Gaussian Mixture Model. As a confounder affecting generative processes of data (feature/label), the latent variables are supposed to capture the discriminative latent distribution and implement data perturbation. Extensive experiments have been conducted on widely-used real imbalanced image datasets. Experimental results demonstrate the superiority of our proposed model by comparing with popular imbalanced classification baselines on imbalance classification task.

Shyness trait is a double-edged personality trait, and could pose a risk in early childhood for later adjustment difficulties. Therefore, it is necessary to pay attention to children with shyness trait and properly supervise them in early education, where the key problem is shyness trait recognition. Although some psychological methods of shyness measurement have been presented, they are high-cost and can only get a one-sided result limited by the targeted subjects. To develop an automated method of shyness trait recognition, we collect a dataset, containing online writing data of 1,754 schoolchildren from an educational website and ground truth labels obtained by a professional scale. The natural implicitness makes shyness trait difficult to be observed from a single point of view and the class imbalanced problem increases the challenge. In this article, a novel shyness trait recognition framework is proposed, which extracts multi-view features of online writing, including document-view, sentence-view and temporality-view ones. Different strategies with different features are applied to each single-view prediction and the multi-view prediction is made by a weighted voting ensemble. To verify the effectiveness, extensive experiments are conducted on the real-world dataset, demonstrating that the multi-view prediction significantly outperforms each single-view prediction and some advanced models of multi-view learning.

Text classification, is one of the key tasks for representing the semantic information of documents, multi-label text classification (MLTC) is an important branch of it. MLTC aims to tag the most relevant labels for the given document. Compared to the standard multi-class case where each document has only one label, it is considerably more difficulty to annotate new coming documents for multi-label text classification. Furthermore, it also suffers from the challenge of highly skewed long-tailed label distribution. i.e., a few labels are associated with a large number of documents (a.k.a. head labels), while a large fraction of labels are associated with a small number of documents (a.k.a. tail labels). Due to the relative infrequency of tail labels, this leads to an imbalance that biases towards predicting more head labels. As challenging as this task is, it is an essential task to tackle since it represents many real-world cases, such as text retrieval of news. To address the challenge, we propose a Triple Alliance Prototype Orthotist Network (TAPON) to build a generic meta-mapping from few-shot prototypes to many-shot classifier parameters, which aims to promote the generalizability of tail classifiers. To be specific, TAPON is a two-stage method. At the first stage focusing on head labels, TAPON obtains the meta-knowledge between many-shot classifier parameters and few-shot prototype of head labels. Head label classifiers are trained by many-shot documents. Meanwhile, the triple alliance prototype is obtained by adopting an Attentive Prototype with the aid of few-shot documents, label semantic information and label correlation. Additionally, a Prototype Orthotist module is especially designed to capture the meta-knowledge between the many-shot classifier and few-shot prototype. At the second stage of transferring, TAPON aims to transfer the generic meta-mapping from head labels to tail labels. It first uses Attentive Prototype to obtain triple alliance prototype for tail labels, and then uses the meta-knowledge obtained from the first stage to get many-shot classifiers for tail labels. By conducting extensive experiments on four benchmark datasets, we show that the proposed TAPON significantly outperforms other state-of-the-art methods for long-tailed multi-label text classification.

Summary matching is a recently proposed paradigm for extractive summarization. It aims to calculate similarities between candidate summaries and their corresponding document and extract summaries by ranking similarities. Due to natural languages often exhibiting the inherent hierarchical structures ingrained with complex syntax and semantics, the latent hierarchical structures between candidate summaries and their corresponding document should be considered when calculating the summary-document similarities. However, the above structural property is hard to model in the Euclidean space. Inspired by the above issues, we explore extractive summarization in the hyperbolic space and propose a new Hyperbolic Siamese Network for the matching-based extractive summarization (HyperSiameseNet). Specifically, HyperSiameseNet projects candidate summaries and their corresponding document representations from the Euclidean space to the Hyperbolic space and then models the summary-document similarities via the squared poincaré distance. Finally, the summary-document similarities are optimized by the margin-based triplet loss for extracting the final summary. The results on the Multi-News dataset have shown the superiority of our model HyperSiameseNet by comparing with the state-of-the-art baselines.

We introduce a novel backbone architecture to improve target-perception ability of feature representation for tracking. Having observed de facto frameworks perform feature matching simply using the backbone outputs for target localization, there is no direct feedback from the matching module to the backbone network, especially the shallow layers. Concretely, only the matching module can directly access the target information, while the representation learning of candidate frame is blind to the reference target. Therefore, the accumulated target-irrelevant interference in shallow stages may degrade the feature quality of deeper layers. In this paper, we approach the problem by conducting multiple branch-wise interactions inside the Siamese-like backbone networks (InBN). The core of InBN is a general interaction modeler (GIM) that injects the target information to different stages of the backbone network, leading to better target-perception of candidate feature representation with negligible computation cost. The proposed GIM module and InBN mechanism are general and applicable to different backbone types including CNN and Transformer for improvements, as evidenced on multiple benchmarks. In particular, the CNN version improves the baseline with 3.2/6.9 absolute gains of SUC on LaSOT/TNL2K. The Transformer version obtains SUC of 65.7/52.0 on LaSOT/TNL2K, which are on par with recent SOTAs.

User behavior data in recommendation are driven by the complex interactions of many intentions behind the user's decision making process. However, user behavior data tends to be sparse because of the limited user response and the vase combinations of users and items, which result in unclear user intentions and suffer from cold-start problem. The intentions are highly compound, and may range from high-level ones that govern user's intrinsic interests and realize the underlying reasons behind the user's decision making processes, to low-level one that characterize a user's extrinsic preference when executing intention to specific items. In this paper, we propose an intention neural process model (INP) for user cold-start recommendation (i.e., user with very few historical interactions), a novel extension of the neural stochastic process family using a general meta learning strategy with intrinsic and extrinsic intention learning for robust user preference learning. By regarding the recommendation process for each user as a stochastic process, INP defines distributions over functions, is capable of rapid adaptation to new users. Our approach learns intrinsic intentions by inferring the high-level concepts associated with user interests or purposes, while capturing the target preference of a user by performing self-supervised intention matching between historical items and target items in a disentangled latent space. Extrinsic intentions are learned by simultaneously generating the point-wise implicit feedback data and creates the pair-wise ranking list by sufficient exploiting both interacted and non-interacted items for each user. Empirical results show that our approach can achieve substantial improvement over the state-of-the-art baselines on cold-start recommendation.

Keyphrase extraction is a fundamental task in natural language processing that aims to extract a set of phrases with important information from a source document.Identifying important keyphrases is the central component of keyphrase extraction, and its main challenge is learning to represent information comprehensively and discriminate importance accurately.In this paper, to address the above issues, we design a new hyperbolic matching model (Hy-perMatch) to explore keyphrase extraction in hyperbolic space.Concretely, to represent information comprehensively, HyperMatch first takes advantage of the hidden representations in the middle layers of RoBERTa and integrates them as the word embeddings via an adaptive mixing layer to capture the hierarchical syntactic and semantic structures.Then, considering the latent structure information hidden in natural languages, HyperMatch embeds candidate phrases and documents in the same hyperbolic space via a hyperbolic phrase encoder and a hyperbolic document encoder.To discriminate importance accurately, HyperMatch estimates the importance of each candidate phrase by explicitly modeling the phrase-document relevance via the Poincaré distance and optimizes the whole model by minimizing the hyperbolic margin-based triplet loss.Extensive experiments are conducted on six benchmark datasets and demonstrate that HyperMatch outperforms the recent state-of-the-art baselines.

Considering the ever-increasing scale of data, which may contain tens of thousands of data points or complicated latent structures, the issue of scalability and algorithmic efficiency becomes of vital importance for clustering. In this paper, we propose cluster-wise amortized mixing coupling processes (AMCP), which is able to achieve efficient amortized clustering in a well-defined non-parametric Bayesian posterior. Specifically, AMCP learns clusters sequentially with the aid of the proposed intra-cluster mixing (IntraCM) and inter-cluster coupling (InterCC) strategies, which investigate the relationship between data points and reference distribution in a linear optimal transport mixing view, and coupling the unassigned set and assigned set to generate new cluster. IntraCM and InterCC avoid pairwise calculation of distances between clusters and reduce the computational complexity from quadratic to linear in the current number of clusters. Furthermore, cluster-wise sequential process is able to improve the quick adaptation ability for the next cluster generation. In this case, AMCP simultaneously learns what makes a cluster, how to group data points into clusters, and how to adaptively control the number of clusters. To illustrate the superiority of the proposed method, we perform experiments on both synthetic data and real-world data in terms of clustering performance and computational efficiency. The source code is available at https://github.com/HuafengHK/AMCP.

Relying on Transformer for complex visual feature learning, object tracking has witnessed the new standard for state-of-the-arts (SOTAs). However, this advancement accompanies by larger training data and longer training period, making tracking increasingly expensive. In this paper, we demonstrate that the Transformer-reliance is not necessary and the pure ConvNets are still competitive and even better yet more economical and friendly in achieving SOTA tracking. Our solution is to unleash the power of multimodal vision-language (VL) tracking, simply using ConvNets. The essence lies in learning novel unified-adaptive VL representations with our modality mixer (ModaMixer) and asymmetrical ConvNet search. We show that our unified-adaptive VL representation, learned purely with the ConvNets, is a simple yet strong alternative to Transformer visual features, by unbelievably improving a CNN-based Siamese tracker by 14.5% in SUC on challenging LaSOT (50.7%$\rightarrow$65.2%), even outperforming several Transformer-based SOTA trackers. Besides empirical results, we theoretically analyze our approach to evidence its effectiveness. By revealing the potential of VL representation, we expect the community to divert more attention to VL tracking and hope to open more possibilities for future tracking beyond Transformer. Code and models are released at https://github.com/JudasDie/SOTS.

Social relations between users have been proven to be a good type of auxiliary information to improve the recommendation performance. However, it is a challenging issue to sufficiently exploit the social relations and correctly determine the user preference from both social and rating information. In this article, we propose a unified Bayesian Additive Matrix Approximation model (BAMA), which takes advantage of rating preference and social network to provide high-quality recommendation. The basic idea of BAMA is to extract social influence from social networks, integrate them to Bayesian additive co-clustering for effectively determining the user clusters and item clusters, and provide an accurate rating prediction. In addition, an efficient algorithm with collapsed Gibbs Sampling is designed to inference the proposed model. A series of experiments were conducted on six real-world social datasets. The results demonstrate the superiority of the proposed BAMA by comparing with the state-of-the-art methods from three views, all users, cold-start users, and users with few social relations. With the aid of social information, furthermore, BAMA has ability to provide the explainable recommendation.

Multi-label text classification (MLTC) aims to annotate documents with the most relevant labels from a number of candidate labels. In real applications, the distribution of label frequency often exhibits a long tail, i.e., a few labels are associated with a large number of documents (a.k.a. head labels), while a large fraction of labels are associated with a small number of documents (a.k.a. tail labels). To address the challenge of insufficient training data on tail label classification, we propose a Head-to-Tail Network (HTTN) to transfer the meta-knowledge from the data-rich head labels to data-poor tail labels. The meta-knowledge is the mapping from few-shot network parameters to many-shot network parameters, which aims to promote the generalizability of tail classifiers. Extensive experimental results on three benchmark datasets demonstrate that HTTN consistently outperforms the state-of-the-art methods. The code and hyper-parameter settings are released for reproducibility.

Xuetao Tian, Liping Jing, Lu He, Feng Liu. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.

Text summarization has been a significant challenge in the Nature Process Language (NLP) field. The approach of dealing with text summarization can be roughly divided into two main paradigms: extractive and abstractive manner. The former allows capturing the most representative snippets in a document while the latter generates a summary by understanding the latent meaning in a material with a language generation model. Recently, studies found that jointly employing the extractive and abstractive summarization models can take advantage of their complementary advantages, creating both concise and informative summaries. However, the reinforced summarization models mainly depend on the ROUGE-based reward, which only has the ability to quantify the extent of word-matching rather than semantic-matching between document and summary. Meanwhile, documents are usually collected with redundant or noisy information due to the existence of repeated or irrelevant information in real-world applications. Therefore, only depending on ROUGE-based reward to optimize the reinforced summarization models may lead to biased summary generation. In this paper, we propose a novel deep \bf{Hy}brid \bf{S}ummarization with semantic weighting \bf{R}eward and latent structure \bf{D}etector (HySRD). Specifically, HySRD introduces a new reward mechanism that simultaneously takes advantage of semantic and syntactic information among documents and summaries. To effectively model the accuracy semantics, a latent structure detector is designed to incorporate the high-level latent structures in the sentence representation for information selection. Extensive experiments have been conducted on two well-known benchmark datasets \emph{CNN/Daily Mail} (short input document) and \emph{BigPatent} (long input document). The automatic evaluation shows that our approach significantly outperforms the state-of-the-art of hybrid summarization models.

In this paper, we propose Cluster-wise Hierarchical Generative Model for deep amortized clustering (CHiGac). It provides an efficient neural clustering architecture by grouping data points in a cluster-wise view rather than point-wise view. CHiGac simultaneously learns what makes a cluster, how to group data points into clusters, and how to adaptively control the number of clusters. The dedicated cluster generative process is able to sufficiently exploit pair-wise or higher-order interactions between data points in both inter- and intra-cluster, which is useful to sufficiently mine the hidden structure among data. To efficiently minimize the generalized lower bound of CHiGac, we design an Ergodic Amortized Inference (EAI) strategy by considering the average behavior over sequence on an inner variational parameter trajectory, which is theoretically proven to reduce the amortization gap. A series of experiments have been conducted on both synthetic and real-world data. The experimental results demonstrated that CHiGac can efficiently and accurately cluster datasets in terms of both internal and external evaluation metrics (DBI and ACC).

Keyphrase extraction is a fundamental task in Natural Language Processing, which usually contains two main parts: candidate keyphrase extraction and keyphrase importance estimation.From the view of human understanding documents, we typically measure the importance of phrase according to its syntactic accuracy, information saliency, and concept consistency simultaneously.However, most existing keyphrase extraction approaches only focus on the part of them, which leads to biased results.In this paper, we propose a new approach to estimate the importance of keyphrase from multiple perspectives (called as KIEMP) and further improve the performance of keyphrase extraction.Specifically, KIEMP estimates the importance of phrase with three modules: a chunking module to measure its syntactic accuracy, a ranking module to check its information saliency, and a matching module to judge the concept (i.e., topic) consistency between phrase and the whole document.These three modules are seamlessly jointed together via an end-toend multi-task learning model, which is helpful for three parts to enhance each other and balance the effects of three perspectives.Experimental results on six benchmark datasets show that KIEMP outperforms the existing state-ofthe-art keyphrase extraction approaches in most cases.

Humans usually explain their reasoning (e.g. classification) by dissecting the image and pointing out the evidence from these parts to the concepts in their minds. Inspired by this cognitive process, several part-level interpretable neural network architectures have been proposed to explain the predictions. However, they suffer from the complex data structure and confusing the effect of the individual part to output category. In this work, an interpretable image recognition deep network is designed by introducing a plug-in transparent embedding space (TesNet) to bridge the high-level input patches (e.g. CNN feature maps) and the out- put categories. This plug-in embedding space is spanned by transparent basis concepts which are constructed on the Grassmann manifold. These basis concepts are enforced to be category-aware and within-category concepts are orthogonal to each other, which makes sure the embedding space is disentangled. Meanwhile, each basis concept can be traced back to the particular image patches, thus they are transparent and friendly to explain the reasoning process. By comparing with state-of-the-art interpretable methods, TesNet is much more beneficial to classification tasks, esp. providing better interpretability on predictions and improve the final accuracy. The code is available at https://github.com/JackeyWang96/TesNet.

User preference modeling in recommendation system aims to improve customer experience through discovering users’ intrinsic preference based on prior user behavior data. This is a challenging issue because user preferences usually have complicated structure, such as inter-user preference similarity and intra-user preference diversity. Among them, inter-user similarity indicates different users may share similar preference, while intra-user diversity indicates one user may have several preferences. In literatures, deep generative models have been successfully applied in recommendation systems due to its flexibility on statistical distributions and strong ability for non-linear representation learning. However, they suffer from the simple generative process when handling complex user preferences. Meanwhile, the latent representations learned by deep generative models are usually entangled, and may range from observed-level ones that dominate the complex correlations between users, to latent-level ones that characterize a user’s preference, which makes the deep model hard to explain and unfriendly for recommendation. Thus, in this paper, we propose an Interpretable Deep Generative Recommendation Model (InDGRM) to characterize inter-user preference similarity and intra-user preference diversity, which will simultaneously disentangle the learned representation from observed-level and latent-level. In InDGRM, the observed-level disentanglement on users is achieved by modeling the user-cluster structure (i.e., inter-user preference similarity) in a rich multimodal space, so that users with similar preferences are assigned into the same cluster. The observed-level disentanglement on items is achieved by modeling the intra-user preference diversity in a prototype learning strategy, where different user intentions are captured by item groups (one group refers to one intention). To promote disentangled latent representations, InDGRM adopts structure and sparsity-inducing penalty and integrates them into the generative procedure, which has ability to enforce each latent factor focus on a limited subset of items (e.g., one item group) and benefit latent-level disentanglement. Meanwhile, it can be efficiently inferred by minimizing its penalized upper bound with the aid of local variational optimization technique. Theoretically, we analyze the generalization error bound of InDGRM to guarantee its performance. A series of experimental results on four widely-used benchmark datasets demonstrates the superiority of InDGRM on recommendation performance and interpretability.

Due to leveraging social relationships between users as well as their past social behavior, social recommendation becomes a core component in recommendation systems. Most existing social recommendation methods only consider direct social relationships among users (e.g., explicit and observed social relations). Recently, researchers proved that indirect social relationships can be effective to improve the recommendation quality when users only have few social connections, because it can identify the user interesting group even though the users have no observed social connection. In the literature, separate two-stage methods are studied, but they cannot explicitly capture the natural relationship between indirect social relations and latent user/item factors. In this paper, the main contribution is to propose a new joint recommendation model taking advantage of the Indirect Social Relations detection and Matrix Factorization collaborative filtering on social network and rating behavior information, which is called as InSRMF. In our work, the user latent factors can simultaneously and seamlessly capture user's personal preferences and social group characteristics. To optimize the InSRMF model, we develop a parallel graph vertex programming algorithm for efficiently handling large scale social recommendation data. Experiments based on four real-world datasets (Ciao, Epinions, Douban and Yelp) are conducted to demonstrate the performance of the proposed model. The experimental results have shown that InSRMF has ability to mine the proper indirect social relations and improve the recommendation performance compared with the testing methods in the literature, especially on the users with few social neighbors, Near-cold-start Users, Pure-cold-start Users and Long-tail Items.

Different from the traditional classification tasks which assume mutual exclusion of labels, hierarchical multi-label classification (HMLC) aims to assign multiple labels to every instance with the labels organized under hierarchical relations. Besides the labels, since linguistic ontologies are intrinsic hierarchies, the conceptual relations between words can also form hierarchical structures. Thus it can be a challenge to learn mappings from word hierarchies to label hierarchies. We propose to model the word and label hierarchies by embedding them jointly in the hyperbolic space. The main reason is that the tree-likeness of the hyperbolic space matches the complexity of symbolic data with hierarchical structures. A new Hyperbolic Interaction Model (HyperIM) is designed to learn the label-aware document representations and make predictions for HMLC. Extensive experiments are conducted on three benchmark datasets. The results have demonstrated that the new model can realistically capture the complex data structures and further improve the performance for HMLC comparing with the state-of-the-art methods. To facilitate future research, our code is publicly available.

Although deep neural networks are effective at extracting high-level features, classification methods usually encode an input into a vector representation via simple feature aggregation operations (e.g.pooling).Such operations limit the performance.For instance, a multi-label document may contain several concepts.In this case, one vector can not sufficiently capture its salient and discriminative content.Thus, we propose Hyperbolic Capsule Networks (HYPERCAPS) for Multi-Label Classification (MLC), which have two merits.First, hyperbolic capsules are designed to capture fine-grained document information for each label, which has the ability to characterize complicated structures among labels and documents.Second, Hyperbolic Dynamic Routing (HDR) is introduced to aggregate hyperbolic capsules in a label-aware manner, so that the label-level discriminative information can be preserved along the depth of neural networks.To efficiently handle large-scale MLC datasets, we additionally present a new routing method to adaptively adjust the capsule number during routing.Extensive experiments are conducted on four benchmark datasets.Compared with the state-of-the-art methods, HY-PERCAPS significantly improves the performance of MLC especially on tail labels.

Discovering hidden pattern from imbalanced data is a critical issue in various real-world applications including computer vision. The existing classification methods usually suffer from the limitation of data especially the minority classes, and result in unstable prediction and low performance. In this paper, a deep generative classifier is proposed to mitigate this issue via both data perturbation and model perturbation. Specially, the proposed generative classifier is modeled by a deep latent variable model where the latent variable aims to capture the direct cause of target label. Meanwhile, the latent variable is represented by a probability distribution over possible values rather than a single fixed value, which is able to enforce uncertainty of model and lead to stable prediction. Furthermore, this latent variable, as a confounder, affects the process of data (feature/label) generation, so that we can arrive at well-justified sampling variability considerations in statistics, and implement data perturbation. Extensive experiments have been conducted on widely-used real imbalanced image datasets. By comparing with the state-of-the-art methods, experimental results demonstrate the superiority of our proposed model on imbalance classification task.

Cross-modal retrieval, given the data of one specific modality as a query, aims to search the relevant data in other modalities. Recently, cross-modal hashing has attracted much attention due to its high efficiency and low storage cost. Its main idea is to approximate the cross-modality similarity via binary codes. This kind of method works well when the cross-modal data is completely observed. However, the real-world application usually avoids this situation, where part of the information is unobserved in some modality. Such partial multimodal data will result in the lack of pairwise information and then destroy the performance of cross-modal hashing. In this paper, we proposed a novel unsupervised cross-modal hashing approach, named as Unsupervised Deep Imputed Hashing (UDIH). It is a two-stage learning strategy. Firstly, the unobserved pairwise data is imputed by the proposed generators. Then a neural network with weighted triplet loss is applied on the correlation graph to learn the hashing code in the Hamming space for each modality, where the correlation graph is constructed with the aid of augmented data. UDIH has the ability to preserve the semantic consistency and difference among data objects. The extensive experimental results have shown that the proposed method outperforms the state-of-the-art methods on two benchmark datasets (MIRFlickr and NUS-WIDE). The source code could be available at https://github.com/AkChen/UDIH.

Deep generative model, especially variational auto-encoder (VAE), has been successfully employed by more and more recommendation systems. The reason is that it combines the flexibility of probabilistic generative model with the powerful non-linear feature representation ability of deep neural networks. The existing VAE-based recommendation models are usually proposed under global assumption by incorporating simple priors, e.g., a single Gaussian, to regularize the latent variables. This strategy, however, is ineffective when the user is simultaneously interested in different kinds of items, i.e., the user’s preference may be highly diverse. In this paper, thus, we propose a Deep Global and Local Generative Model for recommendation to consider both local and global structure among users (DGLGM) under the Wasserstein auto-encoder framework. Besides keeping the global structure like the existing model, DGLGM adopts a non-parametric Mixture Gaussian distribution with several components to capture the diversity of the users’ preferences. Each component is corresponding to one local structure and its optimal size can be determined via the automatic relevance determination technique. These two parts can be seamlessly integrated and enhance each other. The proposed DGLGM can be efficiently inferred by minimizing its penalized upper bound with the aid of local variational optimization technique. Meanwhile, we theoretically analyze its generalization error bounds to guarantee its performance in sparse feedback data with diversity. By comparing with the state-of-the-art methods, the experimental results demonstrate that DGLGM consistently benefits the recommendation system in top-N recommendation task.

Multiview clustering explores complementary information among distinct views to enhance clustering performance under the assumption that all samples have complete information in all available views. However, this assumption does not hold in many real applications, where the information of some samples in one or more views may be missing, leading to partial multiview clustering problems. In this case, significant performance degeneration is usually observed. A collection of partial multiview clustering algorithms has been proposed to address this issue and most treat all different views equally during clustering. In fact, because different views provide features collected from different angles/feature spaces, they might play different roles in the clustering process. With the diversity of different views considered, in this study, a novel adaptive method is proposed for partial multiview clustering by automatically adjusting the contributions of different views. The samples are divided into complete and incomplete sets, while a joint learning mechanism is established to facilitate the connection between them and thereby improve clustering performance. More specifically, the method is characterized by a joint optimization model comprising two terms. The first term mines the underlying cluster structure from both complete and incomplete samples by adaptively updating their importance in all available views. The second term is designed to group all data with the aid of the cluster structure modeled in the first term. These two terms seamlessly integrate the complementary information among multiple views and enhance the performance of partial multiview clustering. Experimental results on real-world datasets illustrate the effectiveness and efficiency of our proposed method.

With the quick development of social websites, there are more opportunities to have different media types (such as text, image, video, etc.) describing the same topic from large-scale heterogeneous data sources. To efficiently identify the inter-media correlations for multimedia retrieval, unsupervised cross-modal hashing (UCMH) has gained increased interest due to the significant reduction in computation and storage. However, most UCMH methods assume that the data from different modalities are well paired. As a result, existing UCMH methods may not achieve satisfactory performance when partially paired data are given only. In this article, we propose a new-type of UCMH method called robust unsupervised cross-modal hashing ( RUCMH ). The major contribution lies in jointly learning modal-specific hash function, exploring the correlations among modalities with partial or even without any pairwise correspondence, and preserving the information of original features as much as possible. The learning process can be modeled via a joint minimization problem, and the corresponding optimization algorithm is presented. A series of experiments is conducted on four real-world datasets (Wiki, MIRFlickr, NUS-WIDE, and MS-COCO). The results demonstrate that RUCMH can significantly outperform the state-of-the-art unsupervised cross-modal hashing methods, especially for the partially paired case, which validates the effectiveness of RUCMH.

As a new occurring unsupervised method, multi-view clustering offers a good way to investigate the hidden structure from multi-view data and attracts extensive attention in the community of machine learning and data mining. One popular approach is to identify a common latent subspace for capturing the multi-view information. However, these methods are still limited due to the unsupervised learning process and suffer from considerable noisy information from different views. To address this issue, we present a novel multi-view subspace clustering method, named self-supervised deep multi-view subspace clustering (\textbf{S2DMVSC}). It seamlessly integrates spectral clustering and affinity learning into a deep learning framework. \textbf{S2DMVSC} has two main merits. One is that the clustering results can be sufficiently exploited to supervise the latent representation learning for each view (via a classification loss) and the common latent subspace learning (via a spectral clustering loss) for multiple views. The other is that the affinity matrix among data objects is automatically computed according to the high-level and cluster-driven representation. Experiments on two scenarios, including original features and multiple hand-crafted features, demonstrate the superiority of the proposed approach over the state-of-the-art baselines.

Interpretability of recommender systems has caused increasing attention due to its promotion of the effectiveness and persuasiveness of recommendation decision, and thus user satisfaction. Most existing methods, such as Matrix Factorization (MF), tend to be black-box machine learning models that lack interpretability and do not provide a straightforward explanation for their outputs. In this paper, we focus on probabilistic factorization model and further assume the absence of any auxiliary information, such as item content or user review. We propose an influence mechanism to evaluate the importance of the users' historical data, so that the most related users and items can be selected to explain each predicted rating. The proposed method is thus called Influencebased Interpretable Recommendation model (In2Rec). To further enhance the recommendation accuracy, we address the important issue of missing not at random, i.e., missing ratings are not independent from the observed and other unobserved ratings, because users tend to only interact what they like. In2Rec models the generative process for both observed and missing data, and integrates the influence mechanism in a Bayesian graphical model. A learning algorithm capitalizing on iterated condition modes is proposed to tackle the non-convex optimization problem pertaining to maximum a posteriori estimation for In2Rec. A series of experiments on four real-world datasets (Movielens 10M, Netflix, Epinions, and Yelp) have been conducted. By comparing with the state-of-the-art recommendation methods, the experimental results have shown that In2Rec can consistently benefit the recommendation system in both rating prediction and ranking estimation tasks, and friendly interpret the recommendation results with the aid of the proposed influence mechanism.

Lin Xiao, Xin Huang, Boli Chen, Liping Jing. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.

Class imbalance is a challenging problem in many real-world applications such as fraudulent transactions detection in finance and diagnosis of rare diseases in medicine, which has attracted more and more attention in the community of machine learning and data mining. The main issue is how to capture the fundamental characteristics of the imbalanced data distribution. In particular, whether the hidden pattern can be truly mined from minority class is still a largely unanswered question after all it contains limited instances. The existing methods provide only a partial understanding of this issue and result in the biased and inaccurate classifiers. To overcome this issue, we propose a novel imbalanced classification framework with two stages. The first stage aims to accurately determine the class distributions by a supervised class distribution learning method under the Wasserstein auto-encoder framework. The second stage makes use of the generative adversarial networks to simultaneously generate instances according to the learnt class distributions and mine the discriminative structure among classes to train the final classifier. This proposed framework focuses on Supervised Class Distribution Learning for Generative Adversarial Networks-based imbalanced classification (SCDL-GAN). By comparing with the state-of-the-art methods, the experimental results demonstrate that SCDL-GAN consistently benefits the imbalanced classification task in terms of several widely-used evaluation metrics on five benchmark datasets.

To handle the large-scale data in terms of storage and searching time, learning to hash becomes popular due to its efficiency and effectiveness in approximate cross-modal nearest neighbors searching. Most existing unsupervised cross-modal hashing methods, to shorten the semantic gap, try to simultaneously minimize the loss of intra-modal similarity and the loss of inter-modal similarity. However, these models can not guarantee in theory these two losses are simultaneously minimized. In this paper, we first theoretically proved that cross-modal hashing could be implemented by protecting both intra-modal and inter-modal similarity with the aid of variational inference technique and point out the problem that maximizing intra and inter-modal similarity is mutually constrained. In this case, we propose an unsupervised cross-modal hashing framework named as Unsupervised Deep Fusion Cross-modal Hashing (UDFCH) which leverages the data fusion to capture the underlying manifold across modalities to avoid above problem. What’s more, in order to reduce the quantization loss, we sample hash codes from different Bernoulli distributions through a reparameterization trick. Our UDFCH framework has two stages. The first stage aims at mining the the intra-modal structure of each modality. The second stage aims to determine the modality-aware hash code by sufficiently considering the correlation and manifold structure among modalities. A series of experiments conducted on three benchmark datasets show that the proposed UDFCH framework outperforms the state-of-the-art methods on different cross-modal retrieval tasks.

Heterogeneous transfer clustering can translate knowledge from some related heterogeneous source domains to the target domain without any supervision. Existing works usually use a large amount of complete co-occurrence data to learn the projection functions mapping heterogeneous data to a common latent feature subspace. However, in many real applications, it is not practical to collect abundant co-occurrence data, while the available co-occurrence data are always incomplete. Another commonly encountered problem is that the complex structure of real heterogeneous data may result in substantial degeneration in clustering performance. To address these issues, we propose a heterogeneous transfer clustering method specifically designed for partial co-occurrence data (HTCPC). It is superior to the existing methods in three facets. First, HTCPC fully uses the partial co-occurrence data in both source and target domains to learn a latent space, maximally extracting useful knowledge for clustering from limited information. Second, it incorporates multi-layer hidden representations, accurately preserving the complex hierarchical structure of data. Third, it enforces approximately orthogonal constraint in representations, effectively characterizing the latent subspace with minimal redundancy. An efficient algorithm has been derived and implemented to realize the proposed HTCPC. A series of experiments on the real datasets have illustrated the advantage of the proposed approach compared with state-of-the-art methods.

In imbalanced data classification, it is hard to learn the hidden pattern from the minority class due to its insufficient information. To solve this problem, a popular type of sampling methods is proposed based on Active Learning framework, but they still suffer from two key issues: how to keep the structure of the original data and avoid imbalance during the learning process. In this paper, we proposed a novel Active Learning framework (COAL) to select and generate informative instances. To keep the structure and enhance the diversity of the original data, we make use of Clustering-based uncertainty sampling to find informative instances. Meanwhile, to avoid imbalance in the active learning process, we make use of Oversampling method to balance the quantities between classes. Extensive experiments have been conducted by using real world datasets with a large range of imbalance ratio (from 2.78 to 66.67). The experimental results show that the proposed COAL outperforms state-of-the-art methods in terms of several well-known evaluation metrics.

Recommender systems offer critical services in the age of mass information. Personalized ranking has been attractive both for content providers and customers due to its ability of creating a user-specific ranking on the item set. Although the powerful factor-analysis methods including latent factor models and deep neural network models have achieved promising results, they still suffer from the challenging issues, such as sparsity of recommendation data, uncertainty of optimization, and etc. To enhance the accuracy and generalization of recommender system, in this paper, we propose a deep generative ranking (DGR) model under the Wasserstein autoencoder framework. Specifically, DGR simultaneously generates the pointwise implicit feedback data (via a Beta-Bernoulli distribution) and creates the pairwise ranking list by sufficient exploiting both interacted and non-interacted items for each user. DGR can be efficiently inferred by minimizing its penalized evidence lower bound. Meanwhile, we theoretically analyze the generalization error bounds of DGR model to guarantee its performance in extremely sparse feedback data. A series of experiments on four large-scale datasets (Movielens (20M), Netflix, Epinions and Yelp in movie, product and business domains) have been conducted. By comparing with the state-of-the-art methods, the experimental results demonstrate that DGR consistently benefit the recommendation system in ranking estimation task, especially for the near-cold-start-users (with less than five interacted items).

According to the sparseness of rating information, the quality of recommender systems has been greatly restricted. In order to solve this problem, much auxiliary information has been used, such as social networks, review information, and item description. Convolutional neural networks (CNNs) have been widely employed by recommender systems, it greatly improved the rating prediction's accuracy especially when combined with traditional recommendation methods. However, a large amount of research focuses on the consistency between the rating-based latent factor and review-based latent factor. But in fact, these two parts are completely different. In this article, the authors propose a model named collaboration matrix factorization (CMF) that combines a projection method with a convolutional matrix factorization (ConvMF) to extract the collaboration between rating-based latent factors and review-based latent factors that comes from the results of the CNN process. Extensive experiments on three real-world datasets show that the projection method achieves significant improvements over the existing baseline.

Sentiment lexicon is a very important resource for opinion mining. Recently, many state-of-the-art works employ deep learning techniques to construct sentiment lexicons. In general, they firstly learn sentiment-aware word embeddings, and then use it as word features to construct sentiment lexicons. However, these methods do not consider the importance of each word to the distinguish of documents' sentiment polarities. As we know, most words among a document do not contribute to understand documents' semantic or sentiment. For example, in the tweet It's a good day, but i can't feel it. I'm really unhappy. The words `unhappy', `feel' and `can't' are much more important than the words `good', `day' in predicting the sentiment polarity of this twitter. Meanwhile, many words, such as `the', `in', `it' and `I'm' are uninformative. In this paper, we propose a novel sparse self-attention LSTM (SSALSTM) to efficiently capture the above intuitive facts, and then construct a large scale sentiment lexicons in twitter. In SSALSTM, we use a novel self-attention mechanism to capture the importance of each words to the distinguish of documents' sentiment polarities. In addition, a $L_1$ regularize is applied in the attentions which can ensure the sparsity characters that most words in a document are semantic and sentiment indistinguishable. Once we learn an efficient sentiment-aware word embedding, we train a classifier which uses sentiment-aware word embedding as features to predict the sentiment polarities of words. Extensive experiments on four publicly available datasets, SemEval 2013-2016, indicate that the sentiment lexicon generated by our proposed model achieves state-of-the-art performance on both supervised and unsupervised sentiment classification tasks.

In this paper, we propose a novel hierarchical supervision topic model to construct a topic-adaptive sentiment lexicon (TaSL) for higher-level classification tasks. It is widely recognized that sentiment lexicon as a useful prior knowledge is crucial in sentiment analysis or opinion mining. However, many existing sentiment lexicons are constructed ignoring the variability of the sentiment polarities of words in different topics or domains. For example, the word “amazing” can refer to causing great surprise or wonder but can also refer to very impressive and excellent. In TaSL, we solve this issue by jointly considering the topics and sentiments of words. Documents are represented by multiple pairs of topics and sentiments, where each pair is characterized by a multinomial distribution over words. Meanwhile, this generating process is supervised under hierarchical supervision information of documents and words. The main advantage of TaSL is that the sentiment polarity of each word in different topics can be sufficiently captured. This model is beneficial to construct a domain-specific sentiment lexicon and then effectively improve the performance of sentiment classification. Extensive experimental results on four publicly available datasets, MR, OMD, semEval13A, and semEval16B were presented to demonstrate the usefulness of the proposed approach. The results have shown that TaSL performs better than the existing manual sentiment lexicon (MPQA), the topic model based domain-specific lexicon (ssLDA), the expanded lexicons(Weka-ED, Weka-STS, NRC, Liu's), and deep neural network based lexicons (nnLexicon, HIT, HSSWE).

With the development of data collection techniques, multi-view clustering becomes an emerging research direction to improve the clustering performance. Existing work has shown that leveraging multi-view information is able to provide a rich and comprehensive description. One of the core problems is how to sufficiently represent multi-view data in the analysis. In this paper, we introduce a tensor-based Representation Learning method for Multi-view Clustering (tRLMvC) that can unify heterogenous and high-dimensional multi-view feature spaces to a low-dimensional shared latent feature space and improve multi-view clustering performance. To sufficiently capture plenty multi-view information, tRLMvC represents multi-view data as a third-order tensor, expresses each tensorial data point as a sparse t-linear combination of all data points with t-product, and constructs a self-expressive tensor through reconstruction coefficients. The low-dimensional multi-view data representation in the shared latent feature space can be obtained via Tucker decomposition on the self-expressive tensor. These two parts are iteratively performed so that the interaction between selfexpressive tensor learning and its factorization can be enhanced and the new representation can be effectively generated for clustering purpose. We conduct extensive experiments on eight multi-view data sets and compare the proposed model with the state-of-the-art methods. Experimental results have shown that tRLMvC outperforms the baselines in terms of various evaluation metrics.

Low-rank matrix approximation (LRMA) has attracted more and more attention in the community of recommendation. Even though LRMA-based recommendation methods (including Global LRMA and Local LRMA) obtain promising results, they suffer from the complicated structure of the large-scale and sparse rating matrix, especially when the underlying system includes a large set of items with various types and a huge amount of users with diverse interests. Thus, they have to predefine the important parameters, such as the rank of the rating matrix and the number of submatrices. Moreover, most existing Local LRMA methods are usually designed in a two-phase separated framework and do not consider the missing mechanisms of rating matrix. In this article, a non-parametric unified Bayesian graphical model is proposed for A daptive Lo cal low-rank M atrix A pproximation ( ALoMA ). ALoMA has ability to simultaneously identify rating submatrices, determine the optimal rank for each submatrix, and learn the submatrix-specific user/item latent factors. Meanwhile, the missing mechanism is adopted to characterize the whole rating matrix. These four parts are seamlessly integrated and enhance each other in a unified framework. Specifically, the user-item rating matrix is adaptively divided into proper number of submatrices in ALoMA by exploiting the Chinese Restaurant Process. For each submatrix, by considering both global/local structure information and missing mechanisms, the latent user/item factors are identified in an optimal latent space by adopting automatic relevance determination technique. We theoretically analyze the model’s generalization error bounds and give an approximation guarantee. Furthermore, an efficient Gibbs sampling-based algorithm is designed to infer the proposed model. A series of experiments have been conducted on six real-world datasets ( Epinions , Douban , Dianping , Yelp , Movielens (10M), and Netflix ). The results demonstrate that ALoMA outperforms the state-of-the-art LRMA-based methods and can easily provide interpretable recommendation results.

In this paper, we propose a Discriminative Semi-Supervised Feature Selection (DSSFS) method. In this method, a ε-dragging technique is introduced to the Rescaled Linear Square Regression in order to enlarge the distances between different classes. An iterative method is proposed to simultaneously learn the regression coefficients, ε-draggings matrix and predicting the unknown class labels. Experimental results show the superiority of DSSFS.

Cross-age face recognition (CAFR) has received more and more attention in real applications, but it is a challenging task due to complex facial aging process. One popular way is modeling CAFR as a traditional face classification problem. However, most of them suffer from one main difficulty: how to effectively extract identity sensitive features that are age insensitive. In this paper, we propose a joint multi-task convolutional neural network (JMCNN) framework. JMCNN consists of two tasks: one for face recognition to learn identity sensitive features (i.e., age-invariant features), the other for age classification to learn age sensitive features, meanwhile, two tasks enhance each other by enforcing a regularization term on two kinds of features. The experimental results on two well-known cross-age datasets (Morph Album 2, CACD) have shown JMCNN is superior to the existing methods.

Review has been proven to be an important information in recommendation. Different from the overall user-item rating matrix, it can provide textual information that exhibits why a user likes an item or not. Recently, more and more researchers have paid attention on review-based rating prediction. There are two challenging issues: how to extract representative features to characterize users / items from reviews and how to leverage them for recommendation system. In this paper, we propose a Neural Gaussian Mixture Model (NGMM) for review-based rating prediction task. Among it, the review textual information is used to construct two parallel neural networks for users and items respectively, so that the users' preferences and items' properties can be sufficiently extracted and represented as two latent vectors. A shared layer is introduced on the top to couple these two networks together and model user-item rating based on the features learned from reviews. Specifically, each rating is modeled via a Gaussian mixture model, where each Gaussian component has zero variance, the mean described by the corresponding component in user's latent vector and the weight indicated by the corresponding component in item's latent vector. Extensive experiments are conducted on five real-world Amazon review datasets. The experimental results have demonstrated that our proposed NGMM model achieves the state-of-the-art performance in review-based rating prediction task.

Recommender system is a popular tool to accurately and actively provide users with potentially interesting information. For capturing the users' preferences and approximating the missing data, matrix completion and approximation are widely adopted. Except for the typical low-rank factorization-based methods, the additive co-clustering approach (ACCAMS) is recently proposed to succinctly approximate large-scale rating matrix. Although ACCAMS efficiently produces effective recommendation result, it still suffers from the cold-start problem. To address this issue, we propose a Social Influence Additive Co-Clustering method (SIACC) by making use of user-item rating data and user-user social relations.

Multi-label problems arise in various domains, including automatic multimedia data categorization, and have generated significant interest in computer vision and machine learning community. However, existing methods do not adequately address two key challenges: exploiting correlations between labels and making up for the lack of labelled data or even missing labelled data. In this paper, we proposed to use a semi-supervised singular value decomposition (SVD) to handle these two challenges. The proposed model takes advantage of the nuclear norm regularization on the SVD to effectively capture the label correlations. Meanwhile, it introduces manifold regularization on mapping to capture the intrinsic structure among data, which provides a good way to reduce the required labelled data with improving the classification performance. Furthermore, we designed an efficient algorithm to solve the proposed model based on the alternating direction method of multipliers, and thus, it can efficiently deal with large-scale data sets. Experimental results for synthetic and real-world multimedia data sets demonstrate that the proposed method can exploit the label correlations and obtain promising and better label prediction results than the state-of-the-art methods.

This paper presents a novel two-phase method for audio representation, discriminative and compact audio representation (DCAR), and evaluates its performance at detecting events and scenes in consumer-produced videos. In the first phase of DCAR, each audio track is modeled using a Gaussian mixture model (GMM) that includes several components to capture the variability within that track. The second phase takes into account both global structure and local structure. In this phase, the components are rendered more discriminative and compact by formulating an optimization problem on a Grassmannian manifold. The learned components can effectively represent the structure of audio. Our experiments used the YLI-MED and DCASE Acoustic Scenes datasets. The results show that variants on the proposed DCAR representation consistently outperform four popular audio representations (mv-vector, i-vector, GMM, and HEM-GMM). The advantage is significant for both easier and harder discrimination tasks; we discuss how these performance differences across tasks follow from how each type of model leverages (or does not leverage) the intrinsic structure of the data.

This paper presents a novel two-phase method for audio representation: Discriminative and Compact Audio Representation (DCAR). In the first phase, each audio track is modeled using a Gaussian mixture model (GMM) that includes several components to capture the variability within that track. The second phase takes into account both global structure and local structure. In this phase, the components are rendered more discriminative and compact by formulating an optimization problem on Grassmannian manifolds, which we found represents the structure of audio effectively. Experimental results on the YLI-MED dataset show that the proposed DCAR representation consistently outperforms state-of-the-art audio representations: i-vector, mv-vector, and GMM.

One of the main research problems in heterogeneous transfer learning is to determine whether a given source domain is effective in transferring knowledge to a target domain, and then to determine how much of the knowledge should be transferred from a source domain to a target domain. The main objective of this paper is to solve this problem by evaluating the relatedness among given domains through transferred weights. We propose a novel method to learn such transferred weights with the aid of co-occurrence data, which contain the same set of instances but in different feature spaces. Because instances with the same category should have similar features, our method is to compute their principal components in each feature space such that co-occurrence data can be rerepresented by these principal components. The principal component coefficients from different feature spaces for the same instance in the co-occurrence data have the same order of significance for describing the category information. By using these principal component coefficients, the Markov Chain Monte Carlo method is employed to construct a directed cyclic network where each node is a domain and each edge weight is the conditional dependence from one domain to another domain. Here, the edge weight of the network can be employed as the transferred weight from a source domain to a target domain. The weight values can be taken as a prior for setting parameters in the existing heterogeneous transfer learning methods to control the amount of knowledge transferred from a source domain to a target domain. The experimental results on synthetic and real-world data sets are reported to illustrate the effectiveness of the proposed method that can capture strong or weak relations among feature spaces, and enhance the learning performance of heterogeneous transfer learning.

Multi-label problems arise in various domains including automatic multimedia data categorization, and have generated significant interest in computer vision and machine learning community. However, existing methods do not adequately address two key challenges: exploiting correlations between labels and making up for the lack of labeled data or even missing labels. In this paper, we proposed a semi-supervised low-rank mapping (SLRM) model to handle these two challenges. SLRM model takes advantage of the nuclear norm regularization on mapping to effectively capture the label correlations. Meanwhile, it introduces manifold regularizer on mapping to capture the intrinsic structure among data, which provides a good way to reduce the required labeled data with improving the classification performance. Furthermore, we designed an efficient algorithm to solve SLRM model based on alternating direction method of multipliers and thus it can efficiently deal with large-scale datasets. Experiments on four real-world multimedia datasets demonstrate that the proposed method can exploit the label correlations and obtain promising and better label prediction results than state-of-the-art methods.

Heterogeneous transfer learning has recently gained much attention as a new machine learning paradigm in which the knowledge can be transferred from source domains to target domains in different feature spaces. Existing works usually assume that source domains can provide accurate and useful knowledge to be transferred to target domains for learning. In practice, there may be noise appearing in given source (text) and target (image) domains data, and thus, the performance of transfer learning can be seriously degraded. In this paper, we propose a robust and non-negative collective matrix factorization model to handle noise in text-to-image transfer learning, and make a reliable bridge to transfer accurate and useful knowledge from the text domain to the image domain. The proposed matrix factorization model can be solved by an efficient iterative method, and the convergence of the iterative method can be shown. Extensive experiments on real data sets suggest that the proposed model is able to effectively perform transfer learning in noisy text and image domains, and it is superior to the popular existing methods for text-to-image transfer learning.

Image label prediction is a critical issue in computer vision and machine learning. In this paper, we propose and develop sparse label-indicator optimization methods for image classification problems. Sparsity is introduced in the label-indicator such that relevant and irrelevant images with respect to a given class can be distinguished. Also, when we deal with multi-class image classification problems, the number of possible classes of a given image can also be constrained to be small in which it is valid for natural images. The resulting sparsity model can be formulated as a convex optimization problem, and it can be solved very efficiently. Experimental results are reported to illustrate the effectiveness of the proposed model, and demonstrate that the classification performance of the proposed method is better than the other testing methods in this paper.

The overabundance of scientific article information has created much inconvenience to researchers seeking interesting articles online. In this paper, we provide a Bi-Relational graph to represent the heterogenous information of scientific article recommendation system, which includes three parts: the article content similarity, researcher interest correlation, and researcher-article readership. Meanwhile, an iterative random walk with restarts learning method is proposed on the Bi-Relational graph to recommend a researcher rating for each article by making use of the known information. The proposed method has ability to perform both old and new article recommendation. A series of experiments on CiteULike dataset have shown that our method is more effective than other testing methods in the paper.

In this paper, we study dictionary learning (DL) approach to identify the representation of low-dimensional subspaces from high-dimensional and nonnegative data. Such representation can be used to provide an affinity matrix among different subspaces for data clustering. The main contribution of this paper is to consider both nonnegativity and sparsity constraints together in DL such that data can be represented effectively by nonnegative and sparse coding coefficients and nonnegative dictionary bases. In the algorithm, we employ the proximal point technique for the resulting DL and sparsity optimization problem. We make use of coding coefficients to perform spectral clustering (SC) for data partitioning. Extensive experiments on real-world high-dimensional and nonnegative data sets, including text, microarray, and image data demonstrate that the proposed method can discover their subspace structures. Experimental results also show that our algorithm is computationally efficient and effective for obtaining high SC performance and interpreting the clustering results compared with the other testing methods.

In this paper, we propose a novel supervised nonnegative matrix factorization-based framework for both image classification and annotation. The framework consists of two phases: training and prediction. In the training phase, two supervised nonnegative matrix factorizations for image descriptors and annotation terms are combined to identify the latent image bases, and to represent the training images in the bases space. These latent bases can capture the representation of the images in terms of both descriptors and annotation terms. Based on the new representation of training images, classifiers can be learnt and built. In the prediction phase, a test image is first represented by the latent bases via solving a linear least squares problem, and then its class label and annotation can be predicted via the trained classifiers and the proposed annotation mapping model. In the algorithm, we develop a three-block proximal alternating nonnegative least squares algorithm to determine the latent image bases, and show its convergent property. Extensive experiments on real-world image data sets suggest that the proposed framework is able to predict the label and annotation for testing images successfully. Experimental results have also shown that our algorithm is computationally efficient and effective for image classification and annotation.

Semi-supervised nonnegative matrix factorization (NMF)receives more and more attention in text mining field. The semi-supervised NMF methods can be divided into two types, one is based on the explicit category labels, the other is based on the pair wise constraints including must-link and cannot-link. As it is hard to obtain the category labels in some tasks, the latter one is more widely used in real applications. To date, all the constrained NMF methods treat the must-link and cannot-link constraints in a same way. However, these two kinds of constraints play different roles in NMF clustering. Thus a novel constrained NMF method is proposed in this paper. In the new method, must-link constraints are used to control the distance of the data in the compressed form, and cannot-ink constraints are used to control the encoding factor. Experimental results on real-world text data sets have shown the good performance of the proposed method.

How best to represent text data is an important problem in text mining tasks including information retrieval, clustering, classification and etc.. In this paper, we proposed a compact document representation with term semantic units which are identified from the implicit and explicit semantic information. Among it, the implicit semantic information is extracted from syntactic content via statistical methods such as latent semantic indexing and information bottleneck. The explicit semantic information is mined from the external semantic resource (Wikipedia). The proposed compact representation model can map a document collection in a low-dimension space (term semantic units which are much less than the number of all unique terms). Experimental results on real data sets have shown that the compact representation efficiently improve the performance of text clustering.

BACKGROUND: In the literature, there are fruitful algorithmic approaches for identification functional modules in protein-protein interactions (PPI) networks. Because of accumulation of large-scale interaction data on multiple organisms and non-recording interaction data in the existing PPI database, it is still emergent to design novel computational techniques that can be able to correctly and scalably analyze interaction data sets. Indeed there are a number of large scale biological data sets providing indirect evidence for protein-protein interaction relationships. RESULTS: The main aim of this paper is to present a prior knowledge based mining strategy to identify functional modules from PPI networks with the aid of Gene Ontology. Higher similarity value in Gene Ontology means that two gene products are more functionally related to each other, so it is better to group such gene products into one functional module. We study (i) to encode the functional pairs into the existing PPI networks; and (ii) to use these functional pairs as pairwise constraints to supervise the existing functional module identification algorithms. Topology-based modularity metric and complex annotation in MIPs will be used to evaluate the identified functional modules by these two approaches. CONCLUSIONS: The experimental results on Yeast PPI networks and GO have shown that the prior knowledge based learning methods perform better than the existing algorithms.

Gene regulatory networks have been long studied in model organisms as a means of identifying functional relationships among genes or their corresponding products. Despite many existing methods for genome-wide construction of such networks, solutions to the gene regulatory networks problem are however not trivial. Here, we present, a hybrid approach with gene expression profiles and gene ontology (HAEO). HAEO makes use of multimethods (overlapping clustering and reverse engineering methods) to effectively and efficiently construct gene regulatory networks from multisources (gene expression profiles and gene ontology). Application to yeast cell cycle dataset demonstrates HAEO's ability to construct validated gene regulatory networks, such as some potential gene regulatory pairs, which cannot be discovered by general inferring methods and identifying cycles (i.e., feedback loops) between genes. We also experimentally study the efficiency of building networks and show that the proposed method, HAEO is much faster than Bayesian networks method.

How best to present query search results is an important problem in search engines and information retrieval systems. When a single query retrieves many results, simply showing them as a long list will provide users with poor overview. Nowadays, ranking and clustering query search results have been two useful separate post-processing techniques to organize retrieved documents. In this paper, we proposed a spectral analysis method based on the content similarity networks to integrate the clustering and ranking techniques for improving literature search. The new approach organizes all these search results into categories intelligently and simultaneously rank the results in each category. A variety of theoretical and empirical studies have demonstrated that the presented method performs well in real applications, especially in biomedical literature retrieval. Moreover, any free text information can be analyzed with the new method, i.e., the proposed approach can be applied to various information systems, such as Web search engines and literature search service.

This paper presents a new k-means type algorithm for clustering high-dimensional objects in sub-spaces. In high-dimensional data, clusters of objects often exist in subspaces rather than in the entire space. For example, in text clustering, clusters of documents of different topics are categorized by different subsets of terms or keywords. The keywords for one cluster may not occur in the documents of other clusters. This is a data sparsity problem faced in clustering high-dimensional data. In the new algorithm, we extend the k-means clustering process to calculate a weight for each dimension in each cluster and use the weight values to identify the subsets of important dimensions that categorize different clusters. This is achieved by including the weight entropy in the objective function that is minimized in the k-means clustering process. An additional step is added to the k-means clustering process to automatically compute the weights of all dimensions in each cluster. The experiments on both synthetic and real data have shown that the new algorithm can generate better clustering results than other subspace clustering algorithms. The new algorithm is also scalable to large data sets.