Few-shot multi-intent spoken language understanding (SLU) aims to detect user’s multiple intents and key slots using a tiny amount of annotated data. Prevailing multi-intent SLU models typically rely on abundant data for effective training, enabling them to capture corresponding relationships between intents and slots. However, in few-shot scenarios, establishing these connections becomes challenging, especially in situations involving multiple intents, which may result in confused relationships between intents and slots. To overcome the challenge, we propose decomposing the multi-label intent detection task into several single-label tasks, which reduces the complexity of model training while preserving the constraint relationships between intents and their related slots. We design description templates for each intent and respectively predict the correlation between the utterance and each intent description while completing the slot filling task under the corresponding intent. Therefore, each intent can independently guide the slot filling process, mitigating potentially confused relationships between multiple intents and slots. Experimental results on public datasets indicate that the performance of our model is better than ChatGPT and achieves state-of-the-art results.

Ancient Chinese poetry (ACP) is a vital component of Chinese traditional culture. Enhancing the performance of related downstream tasks demands the development of high-quality pre-trained language models (PLMs) dedicated to ACP. Notably, the semantics of ACP significantly differ from modern Chinese. Existing PLMs have limited knowledge of ACP and are inadequately aligned with the semantic space of modern Chinese, which constrains the utility for tasks related to ACP. In this paper, we propose a fine-tuning strategy to establish a precise alignment between ACP and modern Chinese semantics on sentence level. This strategy involves the inclusion of corresponding modern Chinese translations alongside original ancient poems, creating a hybrid corpus. This corpus facilitates a more effective transfer of knowledge from existing PLMs to the domain of ACP. Furthermore, we employ a training strategy based on a glyph-based foundational PLM, enabling meticulous fine-tuning. Consequently, we develop a specialized PLM named CP-ChineseBERT. To evaluate the effectiveness of our proposed strategies, we conducted experiments on two real-world datasets, focusing on tasks related to ACP sentiment classification and ACP title prediction. The experimental results demonstrate the significant improvements in performance achieved through our innovative approaches.

Emotion-cause pair extraction (ECPE) aims to obtain all emotion-cause pairs consisting of the emotion clause and the corresponding cause clause in a document. Many existing works for ECPE utilize BERT to obtain representation on each clause in the document, and then perform the classification of Cartesian product among all clause representations or the clause-level sequence tagging. In this paper, we propose to redefine ECPE as the emotion-cause relationship between clauses prediction (ECRP). ECRP fits well with the form of the next sentence prediction task in BERT, which effectively unifies the BERT’s pre-training and the ECPE-specific fi ne-tuning process. According to the task form of ECRP, we reconstruct the original ECPE dataset from the document format to the clause-pair format. The scale of data is effectively expanded, and the imbalance of data is alleviated to a certain extent because some redundant data is filtered out based o n t he r elative distance between clauses. Experiments demonstrate that our ECRP-BERT model outperforms many competitive baselines. Especially in the case of low resources, the ECRP-BERT model still achieves a good performance.

Recent years have witnessed the rapidly growing of the amount of semistructured documents in real-world applications. Due to the huge size of the real-world data, how to manage semistructured documents effectively is a big challenge for researchers. As a fundamental task in natural language processing field, document classification is a feasible way to handle the large-scale semistructured documents. However, existing methods fail to explicitly take advantage of the hierarchical semantics in semistructured documents. It's known that the contained semantics is beneficial for understanding the semistructured documents. Considering the hierarchical structure of a given semistructured document, we propose a semistructured document classification framework which explicitly utilizes the semantic hierarchical attention mechanism. More specifically, the hierarchical attention mechanism and graph neural network are employed to model semistructured documents, by which the multilevel semantic relationships and grammatical information are considered. Moreover, we propose an adaptive class cost learning method to treat the issue of data imbalance. Comprehensive experiments are conducted on two real-world data sets, and the results demonstrate that our framework performs better than selected baselines for semistructured document classification.

Pseudo‐relevance feedback is a well‐studied query expansion technique in which it is assumed that the top‐ranked documents in an initial set of retrieval results are relevant and expansion terms are then extracted from those documents. When selecting expansion terms, most traditional models do not simultaneously consider term frequency and the co‐occurrence relationships between candidate terms and query terms. Intuitively, however, a term that has a higher co‐occurrence with a query term is more likely to be related to the query topic. In this article, we propose a kernel co‐occurrence‐based framework to enhance retrieval performance by integrating term co‐occurrence information into the Rocchio model and a relevance language model (RM3). Specifically, a kernel co‐occurrence‐based Rocchio method (KRoc) and a kernel co‐occurrence‐based RM3 method (KRM3) are proposed. In our framework, co‐occurrence information is incorporated into both the factor of the term discrimination power and the factor of the within‐document term weight to boost retrieval performance. The results of a series of experiments show that our proposed methods significantly outperform the corresponding strong baselines over all data sets in terms of the mean average precision and over most data sets in terms of P@10. A direct comparison of standard Text Retrieval Conference data sets indicates that our proposed methods are at least comparable to state‐of‐the‐art approaches.

Biomedical document classification is a fundamental task in biomedical field. Existing methods do not make full use of the hierarchically semantic structures in biomedical documents which can be utilized to improve the performance of biomedical document classification. In this paper, according to the hierarchical structures in given biomedical documents, we propose two models for biomedical document classification, which are based on the semantically hierarchical attention mechanism. Specifically, we utilize a hierarchical attention mechanism to model biomedical documents, taking into account simultaneously multiple-level semantic relationships in documents. In addition, an adaptive cost sensitive learning method is proposed to address the data imbalance issue. Extensive experiments on two real-world datasets demonstrate the effectiveness of the proposed methods.

At present, spoken language understanding (SLU) in multi-turn dialogue is a research hotspot, which mainly includes intent detection and slot filling. SLU models trained by large-scale corpus can learn good superficial semantic and grammatical information. But they lack the ability for modeling the knowledge needed to understand language. In order to further understand the deep semantic information of the dialogue, external knowledge needs to be modeled and incorporated into the SLU model. In addition, utilizing the correlation between history dialogue and current utterance is able to understand dialogue in multi-turn SLU. Thus, this paper proposes a joint model of intent detection and slot filling based on history context and external knowledge. This model constructs history dialogue encoder to obtain history context. Meanwhile, it constructs knowledge attention over context module. This module selects external knowledge according to the context information in current utterance and obtains knowledge representation. Finally, the history context and external knowledge representation are incorporated into the intent detection and slot filling joint model. The result of experiments on the common dataset demonstrate that with the help of external knowledge and history context, the performance of our model has a significant improvement.

Open source softwares play an important role in information retrieval research. Most of the existing open source information retrieval systems are implemented in Java or C++ programming language. In this paper, we propose Parrot1, a Python-based interactive platform for information retrieval research. The proposed platform has mainly three advantages in comparison with the existing retrieval systems: (1) It is integrated with Jupyter Notebook, an interactive programming platform which has proved to be effective for data scientists to tackle big data and AI problems. As a result, users can interactively visualize and diagnose a retrieval model; (2) As an application written in Python, it can be easily used in combination with the popular deep learning frameworks such as Tersorflow and Pytorch; (3) It is designed especially for researchers. Less code is needed to create a new retrieval model or to modify an existing one. Our efforts have focused on three functionalists: good usability, interactive programming, and good interoperability with the popular deep learning frameworks. To confirm the performance of the proposed system, we conduct comparative experiments on a number of standard test collections. The experimental results show that the proposed system is both efficient and effective, providing a practical framework for researchers in information retrieval.

In probabilistic BM25, term frequency normalization is one of the key components. It is often controlled by parameters $k_1$ and b , which need to be optimized for each given data set. In this paper, we assume and show empirically that term frequency normalization should be specific with query length in order to optimize retrieval performance. Following this intuition, we first propose a new term frequency normalization with query length for probabilistic information retrieval, namely \textttBM25\tiny QL . Then \textttBM25\tiny QL is incorporated into the state-of-the-art models CRTER riptsize 2 and LDA-BM25, denoted as $\textttCRTER riptsize 2 ^\texttt\tiny QL $ and \textttLDA-BM25\tiny QL respectively. A series of experiments show that our proposed approaches \textttBM25\tiny QL , $\textttCRTER riptsize 2 ^\texttt\tiny QL $ and \textttLDA-BM25\tiny QL are comparable to BM25, CRTER riptsize 2 and LDA-BM25 with the optimal b setting in terms of MAP on all the data sets.

Known as phenotypic overlapping, some disease-rel ated symptoms share a common pathologi cal and physiological mechanism. Researchers attempt to visualize the phenotypic relationships between different human diseases from the perspective of machine learning, but traditional visualization methods may be subject to fundamental limitations of metric spaces. Multiple maps t-SNE regularization method, a probabilistic method for visualizing data points in multiple low-dimensional spaces has been proposed to address the limitation. However, the convergence speed is low when apply on the scale dataset. We use the RMSProp with Nesterov momentum method to learn the objective loss function. This method normalize the gradients by applying an exponential moving average of gradient magnitude for each iteration parameter and use Nesterov momentum to counterweigh too high velocities by “peeking ahead” actual objective values in the candidate search direction. This method convergent faster than the original method of convergence speed. Experiments results on several dataset shows that the proposed method outperforms the several version of mm-tSNE with or without regularization, as measured by the neighborhood preservation ratio and error rate. This suggests the modified mm-tSNE regularization can be applied directly in other domain including social, biological and microbiomic datasets.

Traditional information retrieval (IR) models, in which a document is normally represented as a bag of words and their frequencies, capture the term-level and document-level information. Topic models, on the other hand, discover semantic topic-based information among words. In this paper, we consider term-based information and semantic information as two features of query terms and propose a simple enhancement for ad-hoc IR via topic modeling. In particular, three topic-based hybrid models, LDA-BM25, LDA-MATF and LDA-LM, are proposed. A series of experiments on eight standard datasets show that our proposed models can always outperform significantly the corresponding strong baselines over all datasets in terms of MAP and most of datasets in terms of [email protected] and [email protected] A direct comparison on eight standard datasets also indicates our proposed models are at least comparable to the state-of-the-art approaches.

Most of the existing information retrieval models assume that the terms of a text document are independent of each other. These retrieval models integrate three major variables to determine the degree of importance of a term for a document: within document term frequency, document length and the specificity of the term in the collection. Intuitively, the importance of a term for a document is not only dependent on the three aspects mentioned above, but also dependent on the degree of semantic coherence between the term and the document. In this paper, we propose a heuristic approach, in which the degree of semantic coherence of the query terms with a document is adopted to improve the information retrieval performance. Experimental results on standard TREC collections show the proposed models consistently outperform the state-of-the-art models.

In recent years, online social networks are among the most popular websites with high PV (Page View) all over the world, as they have renewed the way for information discovery and distribution. Millions of users have registered on these websites and hence generate formidable amount of user-generated contents every day. The social networks become "giants", likely eligible to carry on any research tasks. However, we have pointed out that these giants still suffer from their "Achilles Heel", i.e., extreme sparsity. Compared with the extremely large data over the whole collection, individual posting documents such as microblogs seem to be too sparse to make a difference under various research scenarios, while actually these postings are different. In this paper we propose to tackle the Achilles Heel of social networks by smoothing the language model via influence propagation. To further our previously proposed work to tackle the sparsity issue, we extend the socialized language model smoothing with bi-directional influence learned from propagation. Intuitively, it is insufficient not to distinguish the influence propagated between information source and target without directions. Hence, we formulate a bi-directional socialized factor graph model, which utilizes both the textual correlations between document pairs and the socialized augmentation networks behind the documents, such as user relationships and social interactions. These factors are modeled as attributes and dependencies among documents and their corresponding users, and then are distinguished on the direction level. We propose an effective learning algorithm to learn the proposed factor graph model with directions. Finally we propagate term counts to smooth documents based on the estimated influence. We run experiments on two instinctive datasets of Twitter and Weibo. The results validate the effectiveness of the proposed model. By incorporating direction information into the socialized language model smoothing, our approach obtains improvement over several alternative methods on both intrinsic and extrinsic evaluations measured in terms of perplexity, nDCG and MAP measurements.

Video popularity prediction plays a foundational role in many aspects of life, such as recommendation systems and investment consulting. Because of its technological and economic importance, this problem has been extensively studied for years. However, four constraints have limited most related works' usability. First, most feature oriented models are inadequate in the social media environment, because many videos are published with no specific content features, such as a strong cast or a famous script. Second, many studies assume that there is a linear correlation existing between view counts from early and later days, but this is not the case in every scenario. Third, numerous works just take view counts into consideration, but discount associated sentiments. Nevertheless, it is the public opinions that directly drive a video's final success/failure. Also, many related approaches rely on a network topology, but such topologies are unavailable in many situations. Here, we propose a Dual Sentimental Hawkes Process (DSHP) to cope with all the problems above. DSHP's innovations are reflected in three ways: (1) it breaks the "Linear Correlation" assumption, and implements Hawkes Process; (2) it reveals deeper factors that affect a video's popularity; and (3) it is topology free. We evaluate DSHP on four types of videos: Movies, TV Episodes, Music Videos, and Online News, and compare its performance against 6 widely used models, including Translation Model, Multiple Linear Regression, KNN Regression, ARMA, Reinforced Poisson Process, and Univariate Hawkes Process. Our model outperforms all of the others, which indicates a promising application prospect.

Buzzwords are the main embodiment of Internet culture, which play an important role in public opinion analysis, social focus tracking and language evolution study. At present, questionnaire has been wildly used as a standard method to obtain network buzzwords, which is subjective and costly. In this paper, we will propose a novel algorithm relying on the time-distribution feature of words and a KL-divergence measure to estimate words' popularity so as to figure out buzzwords in a specific period. The time-distribution feature simply states the fact that buzzwords' usage has a sharp increase during a very short period, which is then modeled formally with the KL-divergence measure. Compared with traditional method involving much workforce, the automatic algorithm presented here is clearly more efficient. Moreover, buzzwords identified in this manner will not be affected by individual's subjective opinions, so they can reflect the language usage in practice better. When applying the algorithm to a social media big data set, our experimental results show that the proposed approach can accurately identify buzzwords in a certain period, which is highly coincident with results tagged manually.

Incorporating semantic information into document representation is effective and potentially significant to improve retrieval performance. Recently, log-bilinear language model (LBL), as a form of neural language model, has been proved to be an effective way to learn semantic word representations, but its feasibility and effectiveness in information retrieval is mostly unknown. In this paper, we study how to efficiently use LBL to improve as-hoc retrieval. We propose a log-bilinear document language model (LB-DM) within the language modeling framework. The key idea is to learn semantically oriented representations for words, and estimate document language models based on these representations. Noise-constrictive estimation is employed to perform fast training on large document collections. Experiment results on standard TREC collections show that LB-DM performs better than translation language model and LDA-based retrieval model.

A main challenge in applying translation language models to information retrieval is how to estimate the 'true' probability that a query could be generated as a translation of a document. The state-of-art methods rely on document-based word co-occurrences to estimate word-word translation probabilities. However, these methods do not take into account the proximity of co-occurrences. Intuitively, the proximity of co-occurrences can be exploited to estimate more accurate translation probabilities, since two words occur closer are more likely to be related. In this paper, we study how to explicitly incorporate proximity information into the existing translation language model, and propose a proximity-based translation language model, called TM-P, with three variants. In our TM-P models, a new concept (proximity-based word co-occurrence frequency) is introduced to model the proximity of word co-occurrences, which is then used to estimate translation probabilities. Experimental results on standard TREC collections show that our TM-P models achieve significant improvements over the state-of-the-art translation models.

In this paper, we deal with two research issues: the automation of visual attribute identification and semantic relation learning between visual attributes and object categories. The contribution is two-fold, firstly, we provide uniform framework to reliably extract both categorical attributes and depictive attributes. Secondly, we incorporate the obtained semantic associations between visual attributes and object categories into a text-based topic model and extract descriptive latent topics from external textual knowledge sources. Specifically, we show that in mining natural language descriptions from external knowledge sources, the relation between semantic visual attributes and object categories can be encoded as Must-Links and Cannot-Links, which can be represented by Dirichlet-Forest prior. To alleviate the workload of manual supervision and labeling in image categorization process, we introduce a semi-supervised training framework using soft-margin semi-supervised SVM classifier. We also show that the large-scale image categorization results can be significantly improved by combining automatically acquired visual attributes. Experimental results show that the proposed model achieves better ability in describing object-related attributes and makes the inferred latent topics more descriptive.

This paper presents a tag-topic model with Dirichlet Forest prior (TTM-DF) for semantic knowledge acquisition from blog. The TTM-DF model extends the tag-topic model (TTM) by replacing the Dirichlet prior with the Dirichlet Forest prior over the topic-word multinomial. The correlation between words are calculated to generate a set of Must-Links and Cannot-Links, then the structures of Dirichlet trees are obtained though encoding the constraints of Must-Links and Cannot-Links. Words under the same subtrees are expected to be more correlated than words under different subtrees. We conduct experiments on a synthetic and a blog dataset. Both of the experimental results show that the TTM-DF model performs much better than the TTM model. It can improve the coherence of the underlying topics and the tag-topic distributions, and capture semantic knowledge effectively.

This paper proposes a novel topic model, Author-Conference Topic-Connection (ACTC) Model for academic network search. The ACTC Model extends the author-conference-topic (ACT) model by adding subject of the conference and the latent mapping information between subjects and topics. It simultaneously models topical aspects of papers, authors and conferences with two latent topic layers: a subject layer corresponding to conference topic, and a topic layer corresponding to the word topic. Each author would be associated with a multinomial distribution over subjects of conference (eg., KM, DB, IR for CIKM 2012), the conference(CIKM 2012), and the topics are respectively generated from a sampled subject. Then the words are generated from the sampled topics. We conduct experiments on a data set with 8,523 authors, 22,487 papers and 1,243 conferences from the well-known Arnetminer website, and train the model with different number of subjects and topics. For a qualitative evaluation, we compare ACTC with three others models LDA, Author-Topic (AT) and ACT in academic search services. Experiments show that ACTC can effectively capture the semantic connection between different types of information in academic network and perform well in expert searching and conference searching.

In this paper, we proposed a perspective Hierarchical Dirichlet Process (pHDP) model to deal with user-tagged image modeling. The contribution is two-fold. Firstly, we associate image features with image tags. Secondly, we incorporate the user's perspectives into the image tag generation process and introduce new latent variables to determine if an image tag is generated from user's perspectives or from the image content. Therefore, the model is able to extract both embedded semantic components and user's perspectives from user-tagged images. Based on the proposed pHDP model, we achieve automatic image tagging with users' perspective. Experimental results show that the pHDP model achieves better image tagging performance compared to state-of-the-art topic models.

The explosive increase of image data on Internet has made it an important, yet very challenging task to index and automatically annotate image data. To achieve that end, sophisticated algorithms and models have been proposed to study the correlation between image content and corresponding text description. Despite the success of previous works, however, researchers are still facing two major difficulties that may undermine their effort of providing reliable and accurate annotations for images. The first difficulty is lacking of comprehensive benchmark image dataset with high quality text descriptions. The second difficulty is lacking of effective way to represent the image content and make it associate with the text descriptions. In our paper, we aim to deal with both problems. To deal with the first problem, we utilize Wikipedia as external knowledge source and enrich the ontology structure of ImageNet database with comprehensive and highly-reliable text descriptions from Wikipedia articles. To address the second problem, we develop a Probabilistic Topic-Connection (PTC) model to represent the connection between latent semantic topic in text description and latent patterns from image feature space. We compare the performance of our model with the currently popular Correspondence LDA (Corr-LDA) model under the same automatic image annotation scenario using cross-validation. Experimental results demonstrate that our model is able to well represent the connection between latent semantic topics and latent patterns in image feature space, thus facilitates knowledge organization and understanding of both image and text descriptions.

In this paper, we propose a new probabilistic generative model, called Topic-Perspective Model, for simulating the generation process of social annotations. Different from other generative models, in our model, the tag generation process is separated from the content term generation process. While content terms are only generated from resource topics, social tags are generated by resource topics and user perspectives together. The proposed probabilistic model can produce more useful information than any other models proposed before. The parameters learned from this model include: (1) the topical distribution of each document, (2) the perspective distribution of each user, (3) the word distribution of each topic, (4) the tag distribution of each topic, (5) the tag distribution of each user perspective, (6) and the probabilistic of each tag being generated from resource topics or user perspectives. Experimental results show that the proposed model has better generalization performance or tag prediction ability than other two models proposed in previous research.

We present NegativeRank, a novel graph-based sentence ranking model to improve the diversity of focused summary by performing random walks over sentence graph with negative edge weights. Unlike the typical eigenvector centrality ranking, our method models the redundancy among sentence nodes as the negative edges. The negative edges can be thought of as the propagation of disapproval votes which can be used to penalize redundant sentences. As the iterative process continues, the initial ranking score of a given node will be adjusted according to a long-term negative endorsement from other sentence nodes. The evaluation results confirm that our proposed method is very effective in improving the diversity of the focused summary, compared to several well-known text summarization methods.