Vision-Language Models (VLMs) leverage large language models (LLMs) to achieve strong performance across various Vision-Language Tasks (VLTs). The semantic gaps between modalities, driven by knowledge differences between sourcedomain pretraining and target-domain multimodal knowledge, are critical obstacles for robust real-world applications. To bridge this gap, we introduce the Multi Level Vision Language Adapter (MLVLA), a novel fine-tuning framework employing a multi-level alignment strategy. Its core innovation lies in multilevel knowledge alignment (i.e., embedding, attention, and token transformation). MLVLA comprises two key components: Knowledge Effective Fine-tuning (KEFT) and Knowledge Gate (KnG). Unlike existing PEFT methods that primarily focus on unimodal adjustments, KEFT specifically targets cross-modal alignment by fine-tuning embedding-level representations to resolve semantic discrepancies between visual and textual features. This alignment ensures that knowledge from different modalities is harmonized for consistent and coherent representation. KnG applies a selfinverse orthogonal matrix at the input of the VLM Transformer blocks and its inverse at its output, preserving semantic axes while adapting token spaces with extreme parameter efficiency. Our experimental results show that MLVLA could be integrated into the open source VLM with minimal architectural intervention, and reveal its full potential in longer training regimes, thereby enhancing multimodal reasoning and domain adaptation.

We propose One-Shot Multi-view Visual Conversational Recommendation (OSMVR), a novel framework to tackle key challenges in personalized recommendation systems from minimal user input. OSMVR addresses three critical issues: the Cold Start problem, the lack of transparent justifications, and the difficulty of accurately estimating nuanced user preferences. Unlike multi-turn systems, OSMVR reduces user burden by structuring the recommendation process into two phases-Preference Estimation and Recommendation Generation-within a single conversational round. It leverages a multi-view approach that enables one-shot preference inference and multi-perspective recommendation generation.

This paper introduces the Lottery Hedge Fund Hypothesis (LHFH) and proposes Knowledge-Aligned Domain Adaptation (KADA), a novel framework for addressing domain discrepancies between source and target domains. While LLMs demonstrate strong generalization across tasks, their performance often degrades on domain-specific tasks requiring specialized knowledge due to a mismatch in domain-specific knowledge distributions.

Our goal is to control text generation with more fine-grained conditions at lower computational cost than is possible with current alternatives; these conditions are attributes (i.e., multiple codes and free-text). As large-scale pre-trained language models (PLMs) offer excellent performance in free-form text generation, we explore efficient architectures and training schemes that can best leverage PLMs. Our framework, Friendly Conditional Text Generator (FCTG), introduces a multi-view attention (MVA) mechanism and two training tasks, Masked Attribute Modeling (MAM) and Attribute Linguistic Matching (ALM), to direct various PLMs via modalities between the text and its attributes. The motivation of FCTG is to map texts and attributes into a shared space, and bridge their modality gaps, as the texts and attributes reside in different regions of semantic space. To avoid catastrophic forgetting, modality-free embedded representations are learnt, and used to direct PLMs in this space, FCTG applies MAM to learn attribute representations, maps them in the same space as text through MVA, and optimizes their alignment in this space via ALM. Experiments on publicly available datasets show that FCTG outperforms baselines over higher level conditions at lower computation cost.

Our goal is to exploit a unified language model so as to explain the generative process of documents precisely in view of their semantic and topic structures. Because various methods model documents in disparate ways, we are motivated by the expectation that coordinating these methods will allow us to achieve this goal more efficiently than using them in isolation; we combine topic models, embedding models, and neural language models. As we focus on the fact that topic models can be shared among, and indeed complement embedding models and neural language models, we propose Word and topic 2 vec (Wat2vec), and Topic Structure-Aware Neural Language Model (TSANL). Wat2vec uses topics as global semantic information and local semantic information as embedding representations of topics and words, and embeds both words and topics in the same space. TSANL uses recurrent neural networks to capture long-range dependencies over topics and words. Since existing topic models demand time consuming learning and have poor scalability, both due to breaking the document?s structure such as order of words and topics, TSANL maintains the orders of words and topics as phrases and segments, respectively. TSANL reduces the calculation cost and required memory by feeding topic recurrent neural networks, and topic specific word networks with these embedding representations. Experiments show that TSANL maintains both segments and topical phrases, and so enhances previous models.

To ease comprehension of given time-stamped corpora, we extend topic models to handle both the specificity and temporality of topics; this is a significant advance over previous models which fail to provide both views simultaneously. Our proposed model consists of the Topic Chronicle Forest(TCF) and Thematic Dirichlet Processes(TDP). TCF is a set of Topic Chronicle Trees, where each tree is a hierarchy of topics that becomes more specialized toward the leaves. Only one tree is defined in each time interval, a region, and is used for TDP to generate a document. The advantage of our approach lies in providing more compact topic organization, while preserving both the semantic of a given corpus and the thematic of each document. Experiments show that TCF is a useful extension for longitudinal topic discovery and tracking, and helps us to organize and digest data sets.

Our proposal, $N$-gram over Context (NOC), is a nonparametric topic model that aims to help our understanding of a given corpus, and be applied to many text mining applications. Like other topic models, NOC represents each document as a mixture of topics and generates each word from one topic. Unlike these models, NOC focuses on both a topic structure as an internal linguistic structure, and N-gram as an external linguistic structure. To improve the quality of topic specific N-grams, NOC reveals a tree of topics that captures the semantic relationship between topics from a given corpus as context, and forms $N$-gram by offering power-law distributions for word frequencies on this topic tree. To gain both these linguistic structures efficiently, NOC learns them from a given corpus in a unified manner. By accessing this entire tree at the word level in the generative process of each document, NOC enables each document to maintain a thematic coherence and form $N$-grams over context. We develop a parallelizable inference algorithm, D-NOC, to support large data sets. Experiments on review articles/papers/tweet show that NOC is useful as a generative model to discover both the topic structure and the corresponding N-grams, and well complements human experts and domain specific knowledge. D-NOC can process large data sets while preserving full generative model performance, by the help of an open-source distributed machine learning framework.

The information overload problem remains serious for both consumers and service/content providers, leading to heightened demands for personalized recommendations. For recommender systems, updating user models is one of the most important tasks to keep up with their changing preferences and trends. Especially since new consumers and items emerge every day, which are promptly rated or reviewed, updating lists of items and rankings is crucial. In this paper, we set the goal of real time recommendation, to present these items instantly. Unlike standard collaborative filtering algorithms, our offline approach focuses only innovative consumers for these predictions, and then uses as few consumers as possible while keeping the same precision. Since innovators exist in many communities, and their opinions will spread and then stimulate their followers to adopt the same behavior, our approach is based on the hypothesis that a set of innova- tive consumers is sufficient to represent the most representative opinions in each community. Following this hypothesis, we derive a scalable method to detect both communities and innovative consumers in each community from a web- scale data from a behavior log. Our evaluation shows that our proposed weighting method can accurately sample given logs, and be compatible only with previous algorithms for real time recommendations.

We propose a Bayesian nonparametric topic model that rep- resents relationships between given labels and the corre- sponding words/phrases, from supervised articles. Unlike existing supervised topic models, our proposal, supervised N-gram topic model (SNT), focuses on both a number of topics and power-law distribution in the word frequencies to extract topic specific N-grams. To achieve this goal, SNT takes a Bayesian nonparametric approach to topic sampling, which generates word distribution jointly with the given variable in textual order, and then form each N-gram word as a hierarchy of Pitman-Yor process priors. Experiments on labeled text data show that SNT is useful as a generative model for discovering more phrases that complement human experts and domain specific knowledge than the existing al- ternatives. The results show that SNT can be applied to various tasks such as automatic annotation.

This paper presents a topic model that discovers the correlation patterns in a given time-stamped document collection and how these patterns evolve over time. Our proposal, the theme chronicle model (TCM) divides traditional topics into temporal and stable topics to detect the change of each theme over time; previous topic models ignore these differences and characterize trends as merely bursts of topics.

This paper presents a topic model that identifies interpretable low dimensional components in time-stamped data for capturing the evolution of trends. Unlike other models for time-stamped data, our proposal, the trend analysis model (TAM), focuses on the difference between temporal words and other words in each document to detect topic evolution over time. TAM introduces a latent trend class variable into each document and a latent switch variable into each token for handling these differences. The trend class has a probability distribution over temporal words, topics, and a continuous distribution over time, where each topic is responsible for generating words. The latter class uses a document specific probabilistic distribution to judge which variable each word comes from for generating words in each token. Accordingly, TAM can explain which topic co-occurrence pattern will appear at any given time, and represents documents of similar content and timestamp as sharing the same trend class. Therefore, TAM projects them on a latent space of trend dimensionality and allows us to predict the temporal evolution of words and topics in document collections. Experiments on various data sets show that the proposed model can capture interpretable low dimensionality sets of topics and timestamps, take advantage of previous models, and is useful as a generative model in the analysis of the evolution of trends.

This paper presents hierarchical topic models for integrating sentiment analysis with collaborative filtering. Our goal is to automatically predict future reviews to a given author from previous reviews. For this goal, we focus on differentiating author's preference, while previous sentiment analysis models process these review articles without this difference. Therefore, we propose a Latent Evaluation Topic model (LET) that infer each author's preference by introducing novel latent variables into author and his/her document layer. Because these variables distinguish the variety of words in each article by merging similar word distributions, LET incorporates the difference of writers' preferences into sentiment analysis. Consequently LET can determine the attitude of writers, and predict their reviews based on like-minded writers' reviews in the collaborative filtering approach. Experiments on review articles show that the proposed model can reduce the dimensionality of reviews to the low-dimensional set of these latent variables, and is a significant improvement over standard sentiment analysis models and collaborative filtering algorithms.

This paper presents a hierarchical generative model that captures the latent relation of cause and effect underlying user behavioral-originated data such as papers, twitter and purchase history. Our proposel, the Latent Interest Topic model (LIT), introduces a latent variable into each document and each author layor in a coherent generative model. We call the former variable the document class, and the latter variable the author class, where these classes are indicator variables that allow the inclusion of different types of probability, and can be shared over documents with similar content and authors with similar interests, respectively. Significantly, unlike other works, LIT differentiates, respectively, document topics and user interests by using these classes. Consequently, LIT is superior to previous models in explaining the causal relationships behind the data by merging similar distributions; it also makes the computation process easier. Experiments on a research paper corpus show that the proposed model can well capture document and author classes, and reduce the dimensionality of documents to a low-dimensional author-document space, making it useful as a generative model.

To realize services that provide serendipity, this paper assesses the surprise of each user when presented recommendations. We propose a recommendation algorithm that focuses on the search time that, in the absence of any recommendation, each user would need to find a desirable and novel item by himself. Following the hypothesis that the degree of user's surprise is proportional to the estimated search time, we consider both innovators' preferences and trends for identifying items with long estimated search times. To predict which items the target user is likely to purchase in the near future, the candidate items, this algorithm weights each item that innovators have purchased and that reflect one or more current trends; it then lists them in order of decreasing weight. Experiments demonstrate that this algorithm outputs recommendations that offer high user/item coverage, a low Gini coefficient, and long estimated search times, and so offers a high degree of recommendation serendipitousness.

This paper presents a hierarchical topic model that simultaneously captures topics and author's interests. Our proposal, the Author Interest Topic model (AIT), introduces a latent variable with a separate probability distribution over topics into each document. Experiments on a research paper corpus show that the AIT is useful as a generative model.

This paper presents a topic model that detects topic distributions over time. Our proposed model, Trend Detection Model (TDM) introduces a latent trend class variable into each document. The trend class has a probability distribution over topics and a continuous distribution over time. Experiments using our data set show that TDM is useful as a generative model in the analysis of the evolution of trends.

This paper proposes a novel Collaborative Filtering scheme; it focuses on the dynamics and precedence of user preference to recommend items that match the latest preference of the target user. In predicting which items this user will purchase in the near future, the proposed algorithm identifies purchase history logs of users who have similar preferences and a high degree of purchase precedence (i.e., purchasing the same items earlier) relative to the target user. We call this metric the Personal Innovator Degree (PID). Experiments using real online sales data show that the proposed method outperforms existing methods.

In this paper, we propose an information retrieval model called Latent Interest Semantic Map (LISM), which features retrieval composed of both Collaborative Filtering(CF) and Probabilistic Latent Semantic Analysis (PLSA). The motivation behind this study is that the relation between users and documents can be explained by the two different latent classes, where users belong probabilistically in one or more classes with the same interest groups, while documents also belong probabilistically in one or more class with the same topic groups. The novel aspect of LISM is that it simultaneously provides a user model and latent semantic analysis in one map. This benefit of LISM is to enable collaborative filtering in terms of user interest and document topic and thus solve the cold start problem.

We propose a novel log analysis method to capture the semantic relations among words appearing in Web search logs. Our method focuses on the reciprocal relations among a user's intentions, stages of information need, and query behavior in seeking information via a search engine. The approach works because it is based on the assumption that a user's intentions in each query can be derived as a model on the basis of his stage of information need and query behavior, through multiple empirical observations of search logs. The user's intentions drive user to change the words in each successive queries and can thus be used to clarify the semantic relations among words. As a result, this method has the advantage of capturing the semantic relations among words without requiring either manual or natural language processing. Our experimental results indicate that semantic relations could successfully be derived from search logs, confirming that an ontology and thesaurus could be constructed automatically.