Multi-image understanding is crucial in real-world applications such as social media analysis and news reporting. However, existing benchmarks fall short in evaluating models' ability to integrate external knowledge and perform cross-image reasoning. To address this gap, we introduce MRBench, a comprehensive benchmark designed to assess knowledge-based reasoning across 12 diverse domains, incorporating four types of image relations: visually similar, identical entities, attribute-associated, and independent images. Additionally, we propose Multimodal Adaptive Retrieval Reasoning (MARR), a novel framework that enables the analysis of relationships among multiple input images and adaptively determines when to terminate the retrieval process. Extensive evaluations of state-of-the-art multimodal large language models (MLLMs) show a notable gap between model and human performance. The best-performing model, Gemini 2.0, reaches 56.86% accuracy, still 20.24% below humans. Proprietary models generally surpass open-source ones, particularly on visually similar and same-entity tasks, underscoring current limits in multi-image reasoning and retrieval and positioning MRBench as a key diagnostic tool. Our benchmark is available for further research and development in this field. https://github.com/Bruce-XJChen/MRBench.

Image-text retrieval stands as a pivotal task within information retrieval, gaining increasing importance with the rapid advancements in Visual-Language Pretraining models. However, current benchmarks for evaluating these models face limitations, exemplified by instances such as BLIP2 achieving near-perfect performance on existing benchmarks. In response, this paper advocates for a more robust evaluation benchmark for image-text retrieval, one that embraces several essential characteristics. Firstly, a comprehensive benchmark should cover a diverse range of tasks in both perception and cognition-based retrieval. Recognizing this need, we introduce ReCoS, a novel benchmark specifically designed for cross-modal image-text retrieval in complex real-life scenarios. Unlike existing benchmarks, ReCoS encompasses 12 retrieval tasks, with a particular focus on three cognition-based tasks, providing a more holistic assessment of model capabilities. To ensure the novelty of the benchmark, we emphasize the use of original data sources, steering clear of reliance on existing publicly available datasets to minimize the risk of data leakage. Additionally, to strike a balance between the complexity of the real world and benchmark usability, ReCoS includes text descriptions that are neither overly detailed, making retrieval overly simplistic, nor under-detailed to the point where retrieval becomes impossible. Our evaluation results shed light on the challenges faced by existing methods, especially in cognition-based retrieval tasks within ReCoS. This underscores the necessity for innovative approaches in addressing the complexities of image-text retrieval in real-world scenarios. Our code and benchmark datasets are available for further research and development in this field https://github.com/Bruce-XJChen/ReCos

Detecting points of interest is a fundamental problem in 3D shape analysis and can be beneficial to various tasks in multimedia processing. Traditional learning-based detection methods usually rely on each vertex’s geometric features to discriminate points of interest from other vertices. Observing that points of interest are related to not only geometric features on themselves but also the geometric features of surrounding vertices, we propose a novel context-aware 3D points of interest detection algorithm by adopting the spatial attention mechanism in this article. By designing a context attention module, our approach presents a novel deep neural network to simultaneously pay attention to the geometric features of vertices and their local contexts during extracting points of interest. To obtain satisfactory extraction results, our method adaptively assigns different weights to those features in a data-driven way. Extensive experimental results on SHREC 2007, SHREC 2011, and SHREC 2014 datasets show that our algorithm achieves superior performance over existing methods.

This letter describes the improvement of two methods of detecting high-frequency oscillations (HFOs) and their use to localize epileptic seizure onset zones (SOZs). The wavelet transform (WT) method was improved by combining the complex Morlet WT with Shannon entropy to enhance the temporal-frequency resolution during HFO detection. And the matching pursuit (MP) method was improved by combining it with an adaptive genetic algorithm to improve the speed and accuracy of the calculations for HFO detection. The HFOs detected by these two methods were used to localize SOZs in five patients. A comparison shows that the improved WT method provides high specificity and quick localization and that the improved MP method provides high sensitivity.