With the rapid growth of accumulated data from various scientific domains, traditional data management systems face challenges in supporting complicated queries, such as pattern search, on massive amounts of data. To serve sophisticated queries through capturing precise features from data, recent data management systems seek to use artificial intelligence (AI) within the querying process. However, the characteristic of AI inference workflow within the querying process, such as intensive computation and expensive requirements for computing resources, becomes a bottleneck of the AI-powered query systems.In this paper, we provide a generalization of AI inference workflow in the context of AI-powered data discovery and we introduce three different caching strategies corresponding to each stage in the AI inference workflow. We provide in-depth performance evaluation on the impact of these caching strategies through a series of strong scaling experiments. Our experimental results show that the AI-powered data querying performance can be significantly improved by applying different caching strategies.

Finding relevant datasets can be a time-consuming and challenging task, especially for self-describing file formats. Current solutions use either exact or partial keyword matching approaches to extract and process metadata queries, but they fail to capture semantic relationships between the metadata content and query keywords. To address this challenge, we introduce PSQS, a novel parallel semantic search method for self-describing files. The method leverages parallel processing and kv2vec semantic similarity measures to retrieve semantically relevant data efficiently. Our evaluation against existing metadata search solutions shows that PSQS offers a new, efficient and effective semantic search functionality for various fields where large self-describing files are used, such as scientific data management, leading to more accurate and efficient data retrieval.