The discourse-level structure of a document can be captured through learning the rhetorical functions of sentences in that document. Existing supervised methods based on pre-trained language models for classifying rhetorical functions of sentences usually focus on utilizing rhetorical words but ignore the topics of sentences. Since topic words can provide additional information for enhancing the learning of the document structure, we present a neural topic model that is integrated with a BERT-based language model through a unified probabilistic generative process for learning both the rhetorical structure and topic structure of documents. For inference, we design a topic attention mechanism to utilize the learned topic words from previous sentences to improve the prediction of the current sentence’s rhetorical label. The extensive experiments on four real-world datasets of different domains show that the proposed model improves the detection of rhetorical functions of sentences and is effective in document modeling and extracting coherent topics.

Latent factor models are widely used in recommender systems. In these models, users and items are represented as vectors in a joint latent factor space. The inner products of user vectors and item vectors are used to model the user-item interactions (e.g., ratings). A review is often posted by the user to explain the given rating. Therefore, reviews can be used to understand how users rate the items and to interpret the latent dimensions of user and item vectors. In this paper, we propose a probabilistic model that learns latent vectors of users and items in a two- or three-dimensional space for visualization. Our proposed model also extracts review topics and visualizes them in the same visualization space for interpreting the ratings. We model the user-item interactions by using the distances between users and items in the visualization space. Extensive experiments using several real-world datasets demonstrate the effectiveness of our proposed model in recommendation and visualization tasks.

We consider the problem of multi-view anomaly detection for multi-view time series data. This task aims to find time steps in time series instances that have inconsistent features across multiple views. To solve the problem, we propose a multi-view deep Markov model that can learn sequential structures in complex high-dimensional multi-view time series data. In our proposed model, each view is modeled by a sequence of latent states and the state transition function is shared across views. Therefore, the inconsistencies in views data will lead to inconsistencies in transitions between latent states across views, which makes the likelihood of abnormal time steps not high. We rely on that property to compute the multi-view anomaly score of each time step in all time series instances. The extensive experiments show that the proposed model is effective in detecting multi-view anomalies in time series data.

Dimensionality reduction aims to find low-dimensional embeddings of high-dimensional data such that the low-dimensional representation preserves some meaningful properties of structures in the original data. When low-dimensional space is 2- or 3-dimensional, the low-dimensional embeddings can be visualized using a scatterplot map. Most of the existing methods try to preserve the local neighborhoods of all data points. However, in general, it is impossible to retain all such information for all data points in the low-dimensional space. As a result, there could be some data points whose neighborhoods are not faithfully displayed in the visualization due to information loss. If the information loss happens around a specific set of points of interest (e.g., specific patients, or proteins under observed), this may be problematic because the withdrawn insights may not be accurate for these observed data points. Therefore, in this paper, we introduce a problem called focused dimensionality reduction where given an original high-dimensional dataset and a set of points of interest, we want to find 2- or 3-dimensional embeddings of the original data such that the information loss in the local neighborhoods surrounding the points of interest is minimized as much as possible. In other words, if the information loss is inevitable, it should not happen around the points of interest. To solve the problem, we extend the stochastic neighbor embedding method and introduce a focused objective function where we put more weight on losses that involve points of interest. Experiments on real-world datasets show that our proposed method is better in preserving the local neighborhood structure of points of interest while the generated visualizations are as good as those generated by the stochastic neighbor embedding method.

Given posts on 'abortion' and posts on 'religion' from a political forum, how can we find topics that are discriminative and those in common? In general, (1) how can we compare and contrast two or more different ('labeled') document collections? Moreover, (2) how can we visualize the data (in 2-d or 3-d) to best reflect the similarities and differences between the collections?

Exploratory analysis of a text corpus is an important task that can be aided by informative visualization. One spatially-oriented form of document visualization is a scatterplot, whereby every document is associated with a coordinate, and relationships among documents can be perceived through their spatial distances. Semantic visualization further infuses the visualization space with latent semantics, by incorporating a topic model that has a representation in the visualization space, allowing users to also perceive relationships between documents and topics spatially. We illustrate how a semantic visualization system called SemVis could be used to navigate a text corpus interactively and topically via browsing and searching.

A document network refers to a data type that can be represented as a graph of vertices, where each vertex is associated with a text document. Examples of such a data type include hyperlinked Web pages, academic publications with citations, and user profiles in social networks. Such data have very high-dimensional representations, in terms of text as well as network connectivity. In this paper, we study the problem of embedding, or finding a low-dimensional representation of a document network that "preserves" the data as much as possible. These embedded representations are useful for various applications driven by dimensionality reduction, such as visualization or feature selection. While previous works in embedding have mostly focused on either the textual aspect or the network aspect, we advocate a holistic approach by finding a unified low-rank representation for both aspects. Moreover, to lend semantic interpretability to the low-rank representation, we further propose to integrate topic modeling and embedding within a joint model. The gist is to join the various representations of a document (words, links, topics, and coordinates) within a generative model, and to estimate the hidden representations through MAP estimation. We validate our model on real-life document networks, showing that it outperforms comparable baselines comprehensively on objective evaluation metrics.

Visualization of high-dimensional data such as text documents is widely applicable. The traditional means is to find an appropriate embedding of the high-dimensional representation in a low-dimensional visualizable space. As topic modeling is a useful form of dimensionality reduction that preserves the semantics in documents, recent approaches aim for a visualization that is consistent with both the original word space, as well as the semantic topic space. In this paper, we address the semantic visualization problem. Given a corpus of documents, the objective is to simultaneously learn the topic distributions as well as the visualization coordinates of documents. We propose to develop a semantic visualization model that approximates L2-normalized data directly. The key is to associate each document with three representations: a coordinate in the visualization space, a multinomial distribution in the topic space, and a directional vector in a high-dimensional unit hypersphere in the word space. We join these representations in a unified generative model, and describe its parameter estimation through variational inference. Comprehensive experiments on real-life text datasets show that the proposed method outperforms the existing baselines on objective evaluation metrics for visualization quality and topic interpretability.

This paper presents our developed general open source for ontology-based information retrieval to answer queries that involve named entities with their ontological features, namely, aliases, classes, and identifiers. We propose a novel approach for semantic search engines that exploit the ontology features of named entities in proximity search and develop an algorithm for computing dynamic distances between named entities and keywords in queries and documents. In particular, it deals with phrase and proximity queries for which the token-based lengths and positions of the queried named entities in a document may vary. The result provides a platform and library for implementing semantic search engines.