Within the Engineering, Procurement, and Construction (EPC) industry, engineers manually create documents based on engineering drawings, which can be time-consuming and prone to human error. For example, the expansion of typical assemblies of instrument items (Instrument Typicals) in Piping and Instrumentation Diagrams (P&IDs) is a labor-intensive task. Each Instrument Typical assembly is depicted in the P&IDs via a simplified representation showing only a subset of the utilized instruments. The expansion activity involves recording all utilized instruments to create an instrument item list document based on the P&IDs for a particular EPC project. Fortunately, Artificial Intelligence (AI) could help to automate this process. In this paper, we propose the first method for automating the process of Instrument Typical expansion in P&IDs. The method utilizes computer vision techniques and domain knowledge rules to extract information about the Instrument Typicals from a project's P&IDs and legend sheets. Subsequently, the extracted information is used to automatically generate the listing of all utilized instruments. The effectiveness of our method is evaluated on P&IDs from large industrial EPC projects, resulting in precision rates exceeding 98% and recall rates surpassing 99%. These results demonstrate the suitability of our method for industrial deployment. The successful application of our method has the potential to reduce engineering costs and increase the efficiency of EPC projects. Furthermore, the method could be adapted for additional applications in the EPC industry, which highlights the method's industrial value.

Interpreting complex engineering drawings requires substantial manual effort in industrial workflows, with engineers spending hundreds of hours verifying correlations between visual elements and structured specifications across thousands of documents. This challenge is particularly acute in the Engineering, Procurement and Construction (EPC) industries, where interpretation errors are propagated into costly procurement and construction mistakes. We propose a cybernetic systems approach using Large Multimodal Models (LMMs) as cognitive partners in engineering documentation workflows, enhancing human capabilities through intelligent assistance in verification and information extraction tasks. To validate this solution, we systematically evaluated five LMMs on 60 piping isometric drawings with varying template structures, measuring both optical character recognition accuracy and correlation capabilities between drawings and their associated Bills of Materials. The results demonstrated significant performance variation between systems, with Claude 3.5 Sonnet and GPT-4o achieving greater accuracy 70%, while open source alternatives faced challenges with complex layouts and ambiguous visual information. These findings establish benchmarks for humanmachine collaboration in engineering documentation processing and provide a framework for integrating intelligent systems into technical workflows across multiple industrial domains.

ABSTRACT Manual analysis of diagrams and legend sheets in engineering projects is time consuming and needs automation. The lack of standardized legend formats complicates creating a general method for automated information extraction. Existing approaches require training and custom rules for each project. This study proposes a novel solution combining optical character recognition with vision language models and multimodal prompt engineering to automate information extraction from diverse legend sheets without training. It integrates legend information with information extracted from diagrams, unlike studies that only focus on diagrams. Our study shows that VLMs, guided by multimodal prompts, can accurately extract information from diverse legend sheets, enabling automatic information extraction in diagrams across engineering projects. We validate our method through a case study involving the extraction of instruments from piping and instrumentation diagrams (P&IDs) and their legends across three projects with varied formats and standards. The proposed method achieved 100% accuracy in legend classification and information extraction, and 99.68% precision and 95.91% recall in generating instrument listings. The results demonstrate the effectiveness of our approach, significantly enhancing the accuracy and efficiency of information extraction from diagrams. This method can be adapted to different legend formats and diagrams, providing a versatile solution for various industries.

The success of any Engineering, Procurement, and Construction (EPC) project depends on the engineering deliverables developed during project execution. An important deliverable is the Line List document, produced by extracting pipeline numbers from Piping and Instrumentation Diagrams (P&IDs). As the creation of this document is time-consuming, the automation of this process could reduce manual engineering work. However, the complexity of the P&IDs renders traditional computer vision approaches unsuitable. Therefore, deep learning text detection could be utilized to achieve this task. This study assessed the applicability of text detection methods for automating pipeline number information extraction in P&IDs. Our findings indicate that the methods previously used to detect text on P&IDs have limitations in accurately capturing the entire line numbers. Furthermore, we propose a line number detection method achieving a recall rate of over 90% on our evaluation data, consisting of P&IDs from diverse industrial projects. Thus, we demonstrate our method's generalizability to different line number formats and its potential for industrial application. Moreover, the proposed method can be adapted to other types of engineering drawings beyond P&IDs. Thus, it could be used in additional applications for digitizing engineering drawings.

Applied Artificial Intelligence (AI) in engineering is gaining significant traction. AI object detection methods can be applied in the engineering industry to extract information from engineering drawings, offering immense benefits to engineers. A promising application of AI in industrial engineering is symbol recognition applied to engineering drawings. However, these drawings often exhibit areas with a high density of symbols, as well as noise in the form of markups, indicating revisions. These factors could cause symbol misclassification or omission, impacting applications reliant on accurate symbol recognition. This study evaluates the accuracy of a symbol recognition model on engineering drawings called Piping and Instrumen-tation Diagrams (P &IDs) exhibiting varying levels of density and markups causing noise. Despite the assumption that density poses a challenge for accurate symbol recognition in engineering drawings, our study reveals that density has no significant impact on recognition performance when a dense detector is employed. In addition, we quantitatively show that markup-induced noise on engineering drawings negatively influences recognition accuracy. Finally, we provide recommendations regarding the applicability of symbol recognition in engineering applications. The study's findings and recommendations apply to any P &IDs, regardless of the standard used, as they were evaluated on various worldwide projects. Moreover, the research not only contributes to the advancement of symbol recognition on P&IDs, but also can be applied to other types of engineering drawings. Thus, it holds the potential for enhancing symbol recognition in various real-world industrial applications and research.

In the Engineering, Procurement, and Construction (EPC) sector, accurate cost estimations during the tendering phase are crucial for maintaining competitiveness, especially with constrained project schedules and rising labor expenses. Typically, these estimations are labor-intensive, relying heavily on manual evaluations of engineering drawings, which are often shared in PDF format due to intellectual property concerns. This study introduces an innovative solution tailored for the energy industry, utilizing Artificial Intelligence (AI) - primarily deep learning (DL) and machine learning (ML) techniques - to streamline material quantity estimation, thereby saving engineering time and costs. Built on empirical data from a large EPC company operating in the energy sector, AI-based product development experiences, and academic research, our approach aims to enhance the efficiency and accuracy of engineering work, promoting better decision-making and resource distribution. While our focus is on enhancing a particular activity within the case company using AI, the method's broader applicability in the EPC sector potentially benefits both industry professionals and researchers. This study not only advances a practical application but also provides valuable insights for those seeking to develop AI -driven solutions across various engineering disciplines.

The Engineering, Procurement and Construction (EPC) businesses operating within the energy sector are recognizing the increasing importance of Artificial Intelligence (AI). Many EPC companies and their clients have realized the benefits of applying AI to their businesses in order to reduce manual work, drive productivity, and streamline future operations of engineered installations in a highly competitive industry. The current AI market offers various solutions and services to support this industry, but organizations must understand how to acquire AI technology in the most beneficial way based on their business strategy and available resources. This paper presents a framework for EPC companies in their transformation towards AI. Our work is based on examples of project execution of AI-based products development at one of the biggest EPC contractors worldwide and on insights from EPC vendor companies already integrating AI into their engineering solutions. The paper covers the entire life cycle of building AI solutions, from initial business understanding to deployment and further evolution. The framework identifies how various factors influence the choice of approach toward AI project development within large international engineering corporations. By presenting a practical guide for optimal approach selection, this paper contributes to the research in AI project management and organizational strategies for integrating AI technology into businesses. The framework might also help engineering companies choose the optimum AI approach to create business value.

Abstract Artificial intelligence is increasingly becoming important to businesses because many companies have realized the benefits of applying machine learning (ML) and deep learning (DL) in their operations. ML and DL have become attractive technologies for organizations looking to automate repetitive tasks to reduce manual work and free up resources for innovation. Unlike rule‐based automation, typically used for standardized and predictable processes, machine learning, especially deep learning, can handle more complex tasks and learn over time, leading to greater accuracy and efficiency improvements. One of such promising applications is to use AI to reduce manual engineering work. This paper discusses a particular case within McDermott where the research team developed a DL model to do a quality check of complex blueprints. We describe the development and the final product of this case—AI‐based software for the engineering, procurement, and construction (EPC) industry that helps to find the design mistakes buried inside very complex engineering drawings called piping and instrumentation diagrams (P&IDs). We also present a cost‐benefit analysis and potential scale‐up of the developed software. Our goal is to share the successful experience of AI‐based product development that can substantially reduce the engineering hours and, therefore, reduce the project's overall costs. The developed solution can also be potentially applied to other EPC companies doing a similar design for complex installations with high safety standards like oil and gas or petrochemical plants because the design errors it captures are common within this industry. It also could motivate practitioners and researchers to create similar products for the various fields within engineering industry.

Artificial intelligence is increasingly becoming important to businesses since many companies have realized the benefits of applying Machine Learning (ML) and Deep Learning (DL) into their operations. Nevertheless, ML/DL technologies' industrial development and deployment examples are still rare and generally confined within a small cluster of large international companies who are struggling to apply ML more broadly and deploy their use cases at a large scale. Meanwhile, current AI market has started offering various solutions and services. Thus, organizations must understand how to acquire AI technology based on their business strategy and available resources. This paper discusses the industrial experience of developing and deploying ML/DL use cases to support organizations in their transformation towards AI. We identify how various factors, like cost, schedule, and intellectual property, can be affected by the choice of approach towards ML/DL project development and deployment within large international engineering corporations. As a research result, we present a framework that covers the trade-offs between those various factors and can support engineering companies to choose the best approach based on their long-term business strategies and, therefore, would help to accomplish their ML/DL project deployment successfully.

AI in the Engineering, Procurement and Construction (EPC) industry has not yet a proven track record in large-scale projects. Since AI solutions for industrial applications became available only recently, deployment experience and lessons learned are still to be built up. Several research papers exist describing the potential of AI, and many surveys and white papers have been published indicating the challenges of AI deployment in the EPC industry. However, there is a recognizable shortage of in-depth studies of deployment experience in academic literature, particularly those focusing on the experiences of EPC companies involved in large-scale project execution with high safety standards, such as the petrochemical or energy sector. The novelty of this research is that we explore in detail the challenges and obstacles faced in developing and deploying AI in a large-scale project in the EPC industry based on real-life use cases performed in an EPC company. Those identified challenges are not linked to specific technology or a company's know-how and, therefore, are universal. The findings in this paper aim to provide feedback to academia to reduce the gap between research and practice experience. They also help reveal the hidden stones when implementing AI solutions in the industry.