Modeling 3D language fields with Gaussian Splatting for open-ended language queries has recently garnered increasing attention. However, recent 3DGS-based models leverage view-dependent 2D foundation models to refine 3D semantics but lack a unified 3D representation, leading to view inconsistencies. Additionally, inherent open-vocabulary challenges cause inconsistencies in object and relational descriptions, impeding hierarchical semantic understanding. In this paper, we propose Hi-LSplat, a view-consistent Hierarchical Language Gaussian Splatting work for 3D open-vocabulary querying. To achieve view-consistent 3D hierarchical semantics, we first lift 2D features to 3D features by constructing a 3D hierarchical semantic tree with layered instance clustering, which addresses the view inconsistency issue caused by 2D semantic features. Besides, we introduce instance-wise and part-wise contrastive losses to capture all-sided hierarchical semantic representations. Notably, we construct two hierarchical semantic datasets to better assess the model's ability to distinguish different semantic levels. Extensive experiments highlight our method's superiority in 3D open-vocabulary segmentation and localization. Its strong performance on hierarchical semantic datasets underscores its ability to capture complex hierarchical semantics within 3D scenes.

Surface defect is a crucial factor affecting the product quality of steel products. Current studies mainly focus on defect recognition and classification using machine vision-based algorithms, which lack the trace of potential causes and the reuse of experiential knowledge. To address this issue, we construct a knowledge graph for steel surface defects by fusing the multi-source and heterogeneous industrial data, including process parameters, chemical compositions, defect images, operation logs and empirical knowledge. A policy-based reinforcement learning approach is developed to solve the path reasoning problem over the industrial knowledge graph in defect detection and diagnosis. The approach employs two agents to explore the path efficiently from opposite directions, utilizes an integrated reward function that comprehensively considers the path direction, path length and entity distance to perform action selection, and adopts the path sharing mechanism and the prior knowledge to update selection policy. Experimental comparisons with the state-of-the-art knowledge reasoning algorithms on two benchmark datasets, NELL-995 and FB15K-237, validate the performance and merits of the proposed method. The effectiveness of the proposed method is also evaluated on a practical steel surface defect dataset, and the results show that our approach performs well in knowledge reasoning on the surface defect graph.Note to Practitioners—The surface quality of products has become a widely concerned focus in manufacturing industries. With the development of industrial IoT and Cyber-physical system technologies, more and more industrial data has been collected, and machine learning-based algorithms have been developed and applied to the recognition of detect defects. However, the algorithms do not take full advantage of the multi-source and heterogeneous defect-related data. On the other hand, it is also difficult to accumulate, inherit and reuse the experts’ knowledge of solving historical cases in the long-term production process. In order to deal with the above obstacles, we apply the knowledge graph for steel surface defect detection. In the proposed approach, a policy-based reinforcement learning algorithm is developed to solve the path reasoning problem over the industrial knowledge graph. To further improve the performance of our algorithm, we employ two agents to explore the path efficiently from opposite directions, utilize an integrated reward function which comprehensively considers the path direction, path length and entity distance to perform action selection, adopt the path sharing mechanism and updated selection policy to reuse the prior knowledge. As a result, our algorithm obtains high precision in knowledge reasoning tasks on two benchmark datasets and a practical steel surface defect dataset compared with some existing algorithms. Hence, it can be readily applied to real surface defect detection problems and facilitates intelligent manufacturing in steel production.

Knowledge graph embedding (KGE) has caught significant interest for its effectiveness in knowledge graph completion (KGC), specifically link prediction (LP), with recent KGE models cracking the LP benchmarks. Despite the rapidly growing literature, insufficient attention was paid to the cooperation between humans and AI on KG. However, humans' capability to analyze graphs conceptually may further improve the efficacy of KGE models with semantic information. To this effect, we carefully designed a human-AI team (HAIT) system dubbed KG-HAIT, which harnesses the human insights on KG by leveraging fully human-designed ad-hoc dynamic programming (DP) on KG to produce human insightful feature (HIF) vectors that capture the subgraph structural feature and semantic similarities. By integrating HIF vectors into the training of KGE models, notable improvements are observed across various benchmarks and metrics, accompanied by accelerated model convergence. Our results underscore the effectiveness of human-designed DP in the task of LP, emphasizing the pivotal role of collaboration between humans and AI on KG. We open avenues for further exploration and innovation through KG-HAlT, paving the way towards more effective and insightful KG analysis techniques.

Medical vision-language pre-training (Med-VLP) models have recently accelerated the fast-growing medical diagnostics application. However, most Med-VLP models learn task-specific representations independently from scratch, thereby leading to great inflexibility when they work across multiple fine-tuning tasks. In this work, we proposeUniDCP, aUnified medical vision-language model withDynamicCross-modal learnablePrompts, which can be plastically applied to multiple medical vision-language tasks within a unified model. Specifically, we explicitly construct a unified framework to harmonize diverse inputs from multiple pre-training tasks by leveraging cross-modal prompts for unification, which accordingly can accommodate heterogeneous medical fine-tuning tasks within a same model. Furthermore, we conceive a dynamic cross-modal prompt optimizing strategy that optimizes the prompts within the shareable space for implicitly processing the shareable clinic knowledge. UniDCP is the first Med-VLP model capable of performing all 8 medical uni-modal and cross-modal tasks over 14 corresponding datasets, consistently yielding superior results over diverse state-of-the-art methods.

The maintenance of power plants highly relies upon precious knowledge and experience of handling faults, which is often stored in reports such as the event report. Simple string matching is the traditional means of retrieving relevant reports, and there is a failure of such methods in understanding the user's search intention properly. With a focus on improving the accuracy of information feedback, this work aims to develop a system of knowledge base question answering. Specifically, natural language processing is employed to improve question comprehension and information retrieval. The BiLSTM-CRF model and the fine-tuned BERT model are used to capture named entities and relations in the query. And the BM25 algorithm and the fine-tuned BERT model are combined to develop a scheme for better information retrieval. On this basis, the application interface of knowledge base question answering towards intelligent power plant maintenance is developed. With this question answering system, power plant operators can have better interaction with the knowledge base and improve collaboration.