As multimedia content continues to grow on the web, the integration of visual and textual data has become a crucial challenge for web applications, particularly in recommendation systems. Large Vision Language Models (LVLMs) have demonstrated considerable potential in addressing this challenge across various tasks that require such multimodal integration. However, their application in multimodal sequential recommendation (MSR) has not been extensively studied. To bridge this gap, we introduce MSRBench, the first comprehensive benchmark designed to systematically evaluate different LVLM integration strategies in web-based recommendation scenarios. We benchmark three state-of-the-art LVLMs, i.e., GPT-4 Vision, GPT-4o, and Claude-3-Opus, on the next item prediction task using the constructed Amazon Review Plus dataset, which includes additional item descriptions generated by LVLMs. Our evaluation examines five integration strategies: using LVLMs as recommender, item enhancer, reranker, and various combinations of these roles. The benchmark results reveal that 1) using LVLMs as rerankers is the most effective strategy, significantly outperforming others that rely on LVLMs to directly generate recommendations or only enhance items; 2) GPT-4o consistently achieves the best performance across most scenarios, particularly when employed as a reranker; 3) the computational inefficiency of LVLMs presents a major barrier to their widespread adoption in real-time multimodal recommendation systems. Our code and datasets are available at https://github.com/PALIN2018/MSRBench.

Infrared small target detection (ISTD) is a challenging task due to the small size and lack of intrinsic features. Meanwhile, small targets in the infrared spectrum often exhibit low contrast, which makes them difficult to distinguish from complex backgrounds. To address these challenges, we propose a novel hierarchical interactive learning network (HIL-Net). Specifically, we design a hierarchical interactive module (HIM), which realizes the hierarchical interaction between low-level and high-level features. By using deep layer information to enhance the expression of lower level features, we are able to better capture the characteristics of small targets. In addition, we introduce the local area enhancement attention (LAEA), which employs feature decomposition and reconstruction to perform fine-grained local contrast calculations, fully utilizing local information and effectively addressing the challenge of low contrast for small infrared targets. Extensive experiments prove that HIL-Net achieves the state-of-the-art results on the public NUAA-SIRST, NUDT-SIRST, IRSTD-1k, and TDSATUA datasets, with the mean intersection over union (mIoU) scores of 77.74%, 88.92%, 68.20%, and 55.18%, respectively.

Recent advancements in large language models (LLMs) have transformed the field of question answering (QA). However, evaluating LLMs in the medical field is challenging due to the lack of standardized and comprehensive datasets. To address this gap, we introduce CMExam, sourced from the Chinese National Medical Licensing Examination. CMExam consists of 60K+ multiple-choice questions for standardized and objective evaluations, as well as solution explanations for model reasoning evaluation in an open-ended manner. For in-depth analyses of LLMs, we invited medical professionals to label five additional question-wise annotations, including disease groups, clinical departments, medical disciplines, areas of competency, and question difficulty levels. Alongside the dataset, we further conducted thorough experiments with representative LLMs and QA algorithms on CMExam. The results show that GPT-4 had the best accuracy of 61.6% and a weighted F1 score of 0.617. These results highlight a great disparity when compared to human accuracy, which stood at 71.6%. For explanation tasks, while LLMs could generate relevant reasoning and demonstrate improved performance after finetuning, they fall short of a desired standard, indicating ample room for improvement. To the best of our knowledge, CMExam is the first Chinese medical exam dataset to provide comprehensive medical annotations. The experiments and findings of LLM evaluation also provide valuable insights into the challenges and potential solutions in developing Chinese medical QA systems and LLM evaluation pipelines.

Air pollution is a serious environmental problem that has attracted much attention. Predicting air pollutant concentration can provide useful information for urban environmental governance decision-making and residents’ daily health control. However, existing methods fail to model the temporal dependencies or have suffer from a weak ability to capture the spatial correlations of air pollutants. In this paper, we propose a general approach to predict air pollutant concentration, named DSTFN, which consists of a data completion component, a similar region selection component, and a deep spatial-temporal fusion network. The data completion component uses tensor decomposition method to complete the missing data of historical air quality. The similar region selection component uses region metadata to calculate the spatial similarity between regions. The deep spatial-temporal fusion network fuses urban heterogeneous data to capture factors affecting air quality and predict air pollutant concentration. Extensive experiments on a real-world dataset demonstrate that our model achieves the highest performance compared with state-of-the-art models for air quality prediction.

Generative Adversarial Networks (GANs) have shown remarkable success in Semantic label map to Photo-realistic image Translation (S2PT) task. However, the results of the state-of-the-art approaches are often limited to blurriness and artifacts, and still far from realistic, since these methods lack effective semantic constrains to preserve the semantic information and ignore the structural correlations between the textures. To address those problems, we propose a SemanticGAN to synthesize high resolution image with fine details and realistic textures from the semantic label map. Specifically, we propose a Semantic Information Preserved Loss (SIPL) to maintain semantic information in the process of the generation via a segmentation model. Furthermore, we develop a novel generator to obtain the correlations between the image textures using newly-designed Correlated Residual Block (CRB). Experiments evaluated on Cityscapes dataset show that SemanticGAN outperforms many recent state-of-the-art methods in terms of qualitative and quantitative performance.

This paper proposes a general approach to predict the spatially fine-grained air quality. The model is based on deep bidirectional and unidirectional long short-term memory (DBU-LSTM) neural network, which can capture bidirectional temporal dependencies and spatial correlations from time series data. Urban heterogeneous data such as point of interest (POI) and road network are used to evaluate the similarities between urban regions. The tensor decomposition method is used to complete the missing historical air quality data of monitoring stations. We evaluate our approach on real data sources obtained in Beijing, and the experimental results show its advantages over baseline methods.

Traffic speed prediction is an important part of intelligent transportation systems (ITS). If road traffic speed is predicted accurately, we can provide not only evidence for urban traffic managers, but also support for other road services such as path planning. Traditional prediction models usually ignore the spatio-temporal dependencies of the traffic dynamics and influences of external factors. This paper proposes a Temporal Graph Convolutional Networks (GTCN) which is composed of spatio-temporal component and external component to solving the traffic speed prediction problem. The spatio-temporal component integrates k-order spectral graph convolution and dilated casual convolution to capture the spatio-temporal dependencies. The external component takes social factors such as day of the week into account. To further improve the prediction accuracy, we consider the road structure features and point of interest (POI) during the construction of the sensor station graph. We evaluate the prediction model on two datasets from the Caltrans Performance Measurement System (CalTrans PeMS). Experiments show that the proposed GTCN model obtains high accuracy and outperforms state-of-the-art baselines.

Along with the proliferation of online digital map and location-based service, very large POI (point of interest) databases have been constructed where a record corresponds to a POI with information including name, category, address, geographical location and other features. A basic spatial query in POI database is POI retrieval. In many scenarios, a user cannot provide enough information to pinpoint the POI except some clue. For example, a user wants to identify a caf é in a city visited many years ago. SHe cannot remember the name and address but she still recalls that "the caf é is about 200 meters away from a restaurant; and turning left at the restaurant there is a bakery 500 meters away, etc.". Intuitively, the clue, even partial and approximate, describes the spatio-textual context around the targeted POI. Motivated by this observation, this work investigates clue-based spatio-textual query which allows user providing clue, i.e., some nearby POIs and the spatial relationships between them, in POI retrieval. The objective is to retrieve k POIs from a POI database with the highest spatio-textual context similarities against the clue. This work has deliberately designed data-quality-tolerant spatio-textual context similarity metric to cope with various data quality problems in both the clue and the POI database. Through crossing valuation, the query accuracy is further enhanced by ensemble method. Also, this work has developed an index called roll-out-star R-tree (RSR-tree) to dramatically improve the query processing efficiency. The extensive tests on data sets from the real world have verified the superiority of our methods in all aspects.

This paper proposes and solves a novel type of spatial queries named Subject-oriented Top-k hot Region (STR) queries. Given a subject S defined by a feature set R and features importance denoted by weights, an STR query retrieves k non-overlapping regions that have the highest scores computed by the number of feature objects and their weights. As an example, the culture subject is defined by exhibition halls, libraries and museums. On the subject, an STR query finds cultural centers intensively distributed feature objects. In this paper, we propose two efficient algorithms, single-partition (SP) algorithm and dual-partition (DP) algorithm, to process STR queries. Extensive experiments evaluate the proposed solutions under a wide range of parameter settings.

Continuously identifying pre-defined patterns in a streaming time series has strong demand in various applications. While most existing works assume the patterns are in equal length and tolerance, this work focuses on the problem where the patterns have various lengths and tolerances, a common situation in the real world. The challenge of this problem roots on the strict space and time requirements of processing the arriving and expiring data in high-speed stream, combined with difficulty of coping with a large number of patterns with various lengths and tolerances. We introduce a novel concept of converging envelope which bounds the tolerance of a group of patterns in various tolerances and equal length and thus dramatically reduces the number of patterns for similarity computation. The basic idea of converging envelope has potential to more general index problems. To index patterns in various lengths and tolerances, we partition patterns into sub-patterns in equal length and an multi-tree index is developed in this paper.