Textual data is a key signal for modeling relationships between news articles in recommendation systems. However, exaggerated and biased texts often hinder the accurate understanding of textual content, negatively affecting recommendation quality. Multimodal approaches have been proposed to address this issue by incorporating associated news images, but they often overlook the complementary role of visual information. To bridge this gap, we propose an Image-Guided Knowledge (IGKnow) distillation framework that transfers image-guided debiasing capability from a multimodal LLM into a text-only LLM. This design allows the distilled model to extract factual, debiased triples from text alone while preserving the benefits of multimodal learning. To further enhance stability, we introduce Triple-level Supervised Fine-Tuning (TriSFT), a permutation-invariant training for triple structures. Moreover, we refine the model through preference alignment to ensure objective, coherent knowledge extraction across news histories. We evaluate the extracted knowledge in sequential news recommendation and show that debiased triples improve recommendation performance.

Session-based recommendation (SBR), which focuses on next-item prediction for anonymous users based on short-term interaction sequences, has garnered increasing attention from researchers. While graph neural networks (GNNs) have become predominant in modeling complex item transition patterns, our empirical study reveals two critical limitations in existing GNN-based SBR methods. On the one hand, they struggle to differentiate between sessions sharing the same last item, resulting in indistinguishable session representations. On the other hand, the inherent popularity bias in session data leads to the over-recommendation of popular items. Inspired by contrastive learning techniques, this paper presents a unified mitigation framework for Same lAst-item confusion in Graph lEarning (SAGE) for SBR. In SAGE, we first obtain normalized session embeddings on constructed session graphs. We then build positive and negative samples of sessions through dual forward propagations and a novel negative sample selection strategy, followed by calculating contrastive loss. Finally, the enhanced session embeddings are utilized for prediction. Extensive experiments on two real-world datasets demonstrate that integrating SAGE with various state-of-the-art GNN-based SBR methods significantly improves their original performances.

Multimodal named entity recognition (MNER) has become a prominent research area for information extraction from social media content. Recent statistics indicate a growing proportion of social media posts containing multiple images. However, most existing methods focus primarily on single-image scenarios. To bridge this gap, we propose a multimodal large language model-assisted denoising multiscale visual fusion framework (LD-MVF) for multiimage MNER (MNER-MI). Our framework follows a coordinated pipeline of filtering, enhancement, and fusion to systematically address the unique challenges of multiimage scenarios. Specifically, to reduce noise from irrelevant images, we introduce an image–text similarity discriminator trained with human feedback, which provides dynamic weights for visual feature aggregation. To enrich the often sparse textual context, we leverage a multimodal large language model (MLLM) to directly generate grounded auxiliary knowledge from the most relevant images, avoiding information loss caused by textual descriptions. Furthermore, we design an adaptive hierarchical fusion module that extracts multiscale visual features from multiple images and injects them as visual prefixes to enhance textual representations through a dynamic gating mechanism. These three components work cooperatively: the discriminator filters out noise to provide cleaner visual input, the MLLM enhances textual semantics with visually-grounded knowledge, and the fusion module integrates multiscale features for fine-grained cross-modal alignment. Extensive experiments demonstrate that LD-MVF achieves state-of-the-art performance on both the MNER-MI and MNER-MI-Plus datasets.

Entity alignment (EA) is crucial for integrating knowledge graphs (KGs) constructed from diverse sources. Conventional unsupervised EA approaches attempt to eliminate human intervention but often suffer from accuracy limitations. With the rise of large language models (LLMs), leveraging their capabilities for EA presents a promising direction. However, it introduces new challenges: formulating the LLM-based EA problem and extracting the background knowledge in LLMs to realize EA without human intervention. This paper proposes HLMEA, a novel hybrid language model-based unsupervised EA method. HLMEA formulates the EA task into a filtering and single-choice problem and synergistically integrates small language models (SLMs) and LLMs. Specifically, SLMs filter candidate entities based on textual representations generated from KG triples. Then, LLMs refine this selection to identify the most semantically aligned entities. An iterative self-training mechanism allows SLMs to distill knowledge from LLM outputs, enhancing the EA ability of hybrid language models in subsequent rounds cooperatively. We also conducted extensive experiments on benchmark datasets to evaluate HLMEA's performance. The results demonstrate that HLMEA significantly outperforms unsupervised and even supervised EA baselines, proving its potential for scalable and effective EA across large KGs. The code and data are available at \url{https://github.com/xnjin-ai/HLMEA}.

Joint information extraction aims to convert unstructured text into structured knowledge by identifying entities and their relations. However, existing methods often rely on static span formation and relation-agnostic validation, limiting their ability to capture dynamic, context-sensitive semantics. We present RePooL, a hierarchical validation framework that performs fine-grained token-level filtering followed by coarse-grained span-level validation, enabling robust multi-granular semantic modeling. RePooL constructs a dual-view knowledge graph that models tokens and relations as distinct node types. It leverages auxiliary structural relations to encode token-relation semantic compatibility via subject and object roles and to compose multi-token spans dynamically, thereby enabling relation-aware validation across multiple granularities. To further strengthen semantic grounding, RePooL incorporates LLM-guided alignment, which evaluates candidate triples against the input text to specifically reinforce coherent extractions. Extensive experiments on standard IE benchmarks show that RePooL achieves superior performance, demonstrating its effectiveness in modeling fine-grained entity-relation interactions.

Conversational recommender systems (CRSs) aim to proactively capture user preferences through natural language dialogue and recommend high-quality items. To achieve this, CRS gathers user preferences via a dialog module and builds user profiles through a recommendation module to generate appropriate recommendations. However, existing CRS faces challenges in capturing the deep semantics of user preferences and dialogue context. In particular, the efficient integration of external knowledge graph (KG) information into dialogue generation and recommendation remains a pressing issue. Traditional approaches typically combine KG information directly with dialogue content, which often struggles with complex semantic relationships, resulting in recommendations that may not align with user expectations.

Link Sign Prediction (LSP) in signed networks is a critical task with applications in recommendation systems, community detection, and social network analysis. Existing methods primarily rely on graph neural networks to exploit structural information, often neglecting the valuable insights from edge-level textual data. Furthermore, utilizing large language models (LLMs) for LSP faces challenges in reliability and interpreting graph structures. To address these issues, we propose a novel STLSP framework that integrates signed networks' \underline{S}tructural and \underline{T}extual information with LLMs for the \underline{LSP} task. STLSP leverages structural balance theory to generate node embeddings that capture positive and negative relationships. These embeddings are transformed into natural language representations through clustering techniques, allowing LLMs to utilize the structural context fully. By integrating these representations with edge text, STLSP improves the accuracy and reliability of the LSP task. Extensive experiments conducted on five real-world datasets demonstrate that STLSP outperformed state-of-the-art baselines, achieving an 8.7% improvement in terms of accuracy. Moreover, STLSP shows robust performance across various LLMs, making it adaptable to different computational environments. The code and data are publically available at https://github.com/sss483/STLSP.

Sequential recommendation has been receiving increasing attention from researchers. Existing sequential recommendation models leverage deep learning models to capture sequential features. However, these methods ignore confounders in the recommendation process, which can lead the model to learn incorrect correlations and fail to accurately capture users' true preferences. Moreover, these methods rely on extensive interaction sequences, but sequential data often suffers from sparsity issues. To address these limitations, this paper proposes a P reference- a ware C ausal I ntervention and Counter f a c tual Data Augmentation ( Pacific ) framework to enhance sequential recommendation. Initially, we model the causal graph of sequential recommendation and categorize user preferences into global long-term preferences, local long-term preferences, and short-term preferences. Then, we introduce the front-door criterion to eliminate the interference of confounders and design different self-attention mechanisms to estimate the causal effects, aiming to capture users' true preferences. In addition, based on counterfactual thinking, we design a counterfactual data augmentation module to generate enriched sequences. Experimental results on four real-world datasets demonstrate the superiority of our proposed approach over state-of-the-art sequential recommendation methods.

Tingxuan Chen, Jun Long, Liu Yang, Zidong Wang, Yongheng Wang, Xiongnan Jin. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2024.

Learning trajectory representations is essential in many Location Based Services (LBS) applications. Most traditional methods extract trajectory representations based on manually defined features, while deep learning-based methods can reduce part of the human effort. We propose a Deep Spatiotemporal Trajectory Clustering (DSTC) framework to tackle the Spatiotemporal Trajectory Representation Learning towards the Clustering-friendly space (STRLC) problem. Solving the STRLC problem is not a trivial task because: (1) Defining a uniform token size for datasets with an uneven density of trajectory data is challenging. (2) Measuring the similarity between trajectories spanning time zero in the time dimension is a problem to be solved. (3) It requires first learning a vector that can represent the overall characteristics of spatiotemporal trajectories and then mapping it to a more suitable space for clustering. To tackle these challenges, we first utilize the density-based clustering method to define tokens representing the trajectory points automatically. Then, we use polar coordinates to represent the temporal dimension of trajectories. Additionally, we improve the learned trajectory representations in a clustering-oriented latent space end to end. Experiments conducted on benchmark datasets demonstrate that DSTC achieves better accuracy than existing methods. Moreover, the representations learned from spatiotemporal trajectory data in the real world can be used to identify popular routes during the day.

Next-POI recommendation aims to explore from user check-in sequence to predict the next possible location to be visited. Existing methods are often difficult to model the implicit association of multi-modal data with user choices. Moreover, traditional methods struggle to fully explore the variation of user preferences at variable time intervals. To tackle these limitations, we propose a Multi-Modal Temporal Knowledge Graph-aware Sub-graph Embedding approach (Mandari). We first construct a novel Multi-Modal Temporal Knowledge Graph. Based on the proposed knowledge graph, we integrate multi-modal information and leverage the graph attention network to calculate sub-graph prediction probability. Next, we implement a temporal knowledge mining method to model the segmentation and periodicity of user check-in and obtain temporal prediction probability. Finally, we fuse temporal prediction probability with the previous sub-graph prediction probability to obtain the final result. Extensive experiments demonstrate that our approach outperforms existing state-of-the-art methods.

A large number of spatial knowledge graphs (SKGs) are available from spatially enriched knowledge bases, e.g., DBpedia and YAGO2. This provides a great chance to understand valuable information about the regions surrounding us. However, it is hard to comprehend SKGs due to the explosively growing volume and the complication of the graph structures. Thus we study the problem of similar region search (SRS), which is an easy-to-use but effective way to explore spatial data. The effectiveness of SRS highly depends on how to measure the region similarity. However, existing approaches cannot make use of the rich information contained in SKGs thus may lead to incorrect results. In this paper, we propose a spatial knowledge representation learning method for region similarity, namely SKRL4RS. SKRL4RS firstly encodes the spatial entities of an SKG into a vector space to make it easier to extract useful features. Then regions are represented by 3-D tensors using the spatial entity embeddings together with geographical information. Finally, region tensors are fed into the conventional triplet network to learn the feature vectors of regions. The region similarity measure learned by SKRL4RS can capture the hierarchical types, semantic relatedness, and relative locations of spatial entities inside a region. Experimental results on two real-world datasets show that our SKRL4RS outperforms the state-of-the-art by a significant margin in terms of the accuracy of measuring region similarity.

Face-pose estimation aims at estimating the gazing direction with two-dimensional face images. It gives important communicative information and visual saliency. However, it is challenging because of lights, background, face orientations, and appearance visibility. Therefore, a descriptive representation of face images and mapping it to poses are critical. In this paper, we use multimodal data and propose a novel face-pose estimation framework named multitask manifold deep learning ($\text{M}^2\text{DL}$). It is based on feature extraction with improved convolutional neural networks (CNNs) and multimodal mapping relationship with multitask learning. In the proposed CNNs, manifold regularized convolutional layers learn the relationship between outputs of neurons in a low-rank space. Besides, in the proposed mapping relationship learning method, different modals of face representations are naturally combined by applying multitask learning with incoherent sparse and low-rank learning with a least-squares loss. Experimental results on three challenging benchmark datasets demonstrate the performance of$\text{M}^2\text{DL}$.

The conventional works on spatial keyword queries for a knowledge base focus on finding a subtree to cover all the query keywords. The retrieved subtree is rooted at a place vertex, spatially close to a query location and compact in terms of the query keywords. However, user requirements may not be satisfied by a single subtree in some application scenarios. A group of subtrees should be combined together to collectively cover the query keywords. In this paper, we propose and study a novel way of searching on a spatial knowledge, namely collective spatial keyword query on a knowledge base (CoSKQ-KB). We formalize the problem of CoSKQ-KB and design a baseline method for CoSKQ-KB (BCK). To further speed up the query processing, an improved scalable method for CoSKQ-KB (iSCK) is proposed based on a set of efficient pruning and early termination techniques. In addition, we conduct empirical experiments on two real-world datasets to show the efficiency and effectiveness of our proposed algorithms.

Recently, the emerging Social Internet of Things (SIoT) has opened up a myriad of research opportunities. One of the most fundamental research challenges posed by SIoT and its objective is to allow smart objects to create and maintain their own social networks. In this paper, we are interested in constructing these social networks, which are built upon social relationships between smart objects, and quantitatively estimating the social strength values of the relations. In particular, we propose a model named social strength prediction model, which infers social connections among smart objects and predicts the strength of the connections using the co-usage data of the objects. The proposed model is divided into two major components: 1) entropy-based and 2) distance-based social strength computations. The two components capture different properties of co-usages of objects, namely, the diversity and spatiotemporal features, which are all essential factors that contribute to the values of social strength. In order to test the feasibility of the proposed model, we conducted extensive sets of experiments with real-world datasets, which include the history data of various object usages. Based on the results, we show that the proposed model is in fact capable of inferring social connections and predicting their corresponding social strength with relatively high precision and recall.

In the landscape of Web of Things (WoT), we encounter various WoT platforms. The WoT platforms can collect, manage and mash up a huge mass of things and their data by using Web technologies. Considering heterogeneous things deployed across different platforms, the interoperability issue between things and platforms has been raised as one of major challenges in achieving the vision of the Internet of Things (IoT). In this paper, we propose a Semantic Service Description (SSD) ontology to guarantee the semantic consistency of a service description. We also present an approach to generating a service description and deploying a set of heterogeneous things to different platforms automatically. Through the implementation of a prototype, the applicability of our proposal is demonstrated and the experimental results are discussed.

To efficiently explore heterogeneous objects on the Internet and uniquely identify them, we propose a scalable physical-object naming system that reuses the existing ontologies and assigns URL-based semantic identifiers. We implement a prototype of the proposed system and evaluate its performance using synthetically generated data based on the W3C Semantic Sensor Network ontology.