Despite significant progress in text anomaly detection for web applications such as spam filtering and fake news detection, existing methods are fundamentally limited to document-level analysis, unable to identify which specific parts of a text are anomalous. We introduce token-level anomaly detection, a novel paradigm that enables fine-grained localization of anomalies within text. We formally define text anomalies at both document and token-levels, and propose a unified detection framework that operates across multiple levels. To facilitate research in this direction, we collect and annotate three benchmark datasets spanning spam, reviews and grammar errors with token-level labels. Experimental results demonstrate that our framework achieves better performance than other 6 baselines, opening new possibilities for precise anomaly localization in text. All the codes and data are publicly available on https://github.com/charles-cao/TokenCore.

Predicting click-through rates is crucial in various fields, including online advertising and recommendation systems. The key to improving the performance of CTR prediction lies in learning a robust user representation, particularly by analyzing their historical behaviors. Previous studies usually model behavior sequences through attention-based sequence models or graph-based methods, which usually struggle to explore diverse latent interests or accurately model user behaviors. Moreover, this challenge is exacerbated when users' historical behaviors are sparse, a common issue in real-world business-to-business (B2B) e-commerce scenarios. In this paper, we propose a novel Graph-Enhanced Interest Network (GEIN) to capture users' latent intents and facilitate the sequential learning of sparse behavior sequences. Specifically, we first construct a hierarchical item-intent heterogeneous graph to enrich the representation of sparse behaviors using diverse information from graphs. Next, we build a user-level behavior interest factor graph to accurately capture user interests. Additionally, a contrastive learning mechanism is incorporated to mitigate the negative robustness impacts caused by sparsity. Extensive experiments on real-world datasets demonstrate that our proposed GEIN outperforms a wide range of state-of-the-art methods. Furthermore, online A/B testing also confirms the superiority of GEIN over competing baselines in a real-world production environment.

Current methods for few-shot action recognition mainly fall into the metric learning framework following ProtoNet, which demonstrates the importance of prototypes. Although they achieve relatively good performance, the effect of multimodal information is ignored, e.g. label texts. In this work, we propose a novel MultimOdal PRototype-ENhanced Network (MORN), which uses the semantic information of label texts as multimodal information to enhance prototypes. A CLIP visual encoder and a frozen CLIP text encoder are introduced to obtain features with good multimodal initialization. Then in the visual flow, visual prototypes are computed by a visual prototype-computed module. In the text flow, a semantic-enhanced (SE) module and an inflating operation are used to obtain text prototypes. The final multimodal prototypes are then computed by a multimodal prototype-enhanced (MPE) module. Besides, we define a PRototype SImilarity DiffErence (PRIDE) to evaluate the quality of prototypes, which is used to verify our improvement on the prototype level and effectiveness of MORN. We conduct extensive experiments on four popular few-shot action recognition datasets: HMDB51, UCF101, Kinetics and SSv2, and MORN achieves state-of-the-art results. When plugging PRIDE into the training stage, the performance can be further improved.

The recommendation has been playing a key role in many industries, e.g., e-commerce, streaming media, social media, etc. Recently, a new recommendation scenario, called Trigger-Induced Recommendation (TIR), where users are able to explicitly express their instant interests via trigger items, is emerging as an essential role in many e-commerce platforms, e.g., Alibaba.com and Amazon. Without explicitly modeling the user's instant interest, traditional recommendation methods usually obtain sub-optimal results in TIR. Even though there are a few methods considering the trigger and target items simultaneously to solve this problem, they still haven't taken into account temporal information of user behaviors, the dynamic change of user instant interest when the user scrolls down and the interactions between the trigger and target items. To tackle these problems, we propose a novel method -- Deep Evolutional Instant Interest Network (DEI2N), for click-through rate prediction in TIR scenarios. Specifically, we design a User Instant Interest Modeling Layer to predict the dynamic change of the intensity of instant interest when the user scrolls down. Temporal information is utilized in user behavior modeling. Moreover, an Interaction Layer is introduced to learn better interactions between the trigger and target items. We evaluate our method on several offline and real-world industrial datasets. Experimental results show that our proposed DEI2N outperforms state-of-the-art baselines. In addition, online A/B testing demonstrates the superiority over the existing baseline in real-world production environments.

Zero-shot entity retrieval, aiming to link mentions to candidate entities under the zero-shot setting, is vital for many tasks in Natural Language Processing. Most existing methods represent mentions/entities via the sentence embeddings of corresponding context from the Pre-trained Language Model. However, we argue that such coarse-grained sentence embeddings can not fully model the mentions/entities, especially when the attention scores towards mentions/entities are relatively low. In this work, we propose GER, a Graph enhanced Entity Retrieval framework, to capture more fine-grained information as complementary to sentence embeddings. We extract the knowledge units from the corresponding context and then construct a mention/entity centralized graph. Hence, we can learn the fine-grained information about mention/entity by aggregating information from these knowledge units. To avoid the graph bottleneck for the central mention/entity node, we construct a hierarchical graph and design a novel Hierarchical Graph Attention Network~(HGAN). Experimental results on popular benchmarks demonstrate that our proposed GER framework performs better than previous state-of-the-art models.

Chinese Spelling Correction (CSC) is gaining increasing attention in recent years. Despite its extensive use in many applications, such as search engine and optical character recognition system, little has been explored in medical scenarios in which complex and uncommon medical entities are easily misspelled. Correcting the misspellings of medical entities is arguably more difficult than those in the open domain due to its requirements of specific domain knowledge. In this work, we define the task of Medical-domain Chinese Spelling Correction (MCSC) and propose MCSCSet, a large-scale specialist-annotated dataset that contains about 200k samples. In contrast to existing open-domain CSC datasets, MCSCSet involves: i) extensive real-world medical queries collected from Tencent Yidian, ii) corresponding misspelled sentences manually annotated by medical specialists. Our work further offers a medical-domain confusion set consisting of the common error-prone characters in medicine and their corresponding misspellings. Extensive empirical studies have shown significant gaps between the open-domain and medical-domain spelling correction, highlighting the need to develop high-quality datasets that allow for CSC in specific domains. Moreover, our work benchmarks several representative methods, establishing baselines for future work.

Generating natural language descriptions for knowledge graph (KG) is an important category for intelligent writing. Recent models on this task substitute the sequence encoder in a commonly used encoder-decoder framework with a graph encoder. However, these models suffer from entity missing and repetition. In this paper, we propose a novel end-to-end generation model named G2T, which integrates a novel Graph Structure Enhanced Mechanism (GSEM) and a Copy Coverage Loss (CCL). Instead of just considering graph structure in the encoding phase in most existing methods, our GSEM fully utilizes graph structure in the decoding phase and helps to mitigate entity missing problem. Moreover, our CCL can further improve performance by avoiding generating repeated entities. With their help, our model is capable of generating fluent description for KG. The results of automatic and human evaluations show that our model outperforms the state-of-the-art models.

Analyzing public opinions towards products, services and social events is an important but challenging task. Despite the remarkable successes of deep neural networks in sentiment analysis, these approaches do not make full use of the prior sentiment knowledge (e.g., sentiment lexicon, negation words, intensity words). In this paper, we propose a Sentiment-Aware Attention Network (SAAN) to boost the performance of sentiment analysis, which adopts a three-step strategy to learn the sentiment-specific sentence representation. First, we employ a word-level mutual attention mechanism to model word-level correlation. Next, a phrase-level convolutional attention is designed to obtain phrase-level correlation. Finally, a sentence-level multi-head attention mechanism is proposed to capture various sentimental information from different subspaces. The experiments on Movie Review (MR) and Stanford Sentiment Treebank (SST) show that our model consistently outperform the previous methods for sentiment analysis.

Nonnegative Matrix Factorization (NMF) has been proven to be effective in text mining. However, since NMF is a well-known unsupervised components analysis technique, the existing NMF method can not deal with prior constraints, which are beneficial to clustering or classification tasks. In this paper, we address the text clustering problem via a novel strategy, called Pairwise Constraintsguided Non-negative Matrix Factorization (PCNMF for short). Differing from the traditional NMF method, the proposed method can capture the available abundance prior constraints in original space, which result in more effective for clustering or information retrieval. Therefore, PCNMF enforces the discriminative capability in the reduced space. Utilizing the appropriate transformation, PCNMF represents as a new optimization problem, which can be efficiently solved by an iterative approach. The cluster membership of each document can be easily determined as the standard NMF. Empirical studies based on Benchmark document corpus demonstrate appealing results.