Digital twin (DT) is one of the most promising enabling technologies for realizing smart manufacturing and Industry 4.0. DTs are characterized by the seamless integration between the cyber and physical spaces. The importance of DTs is increasingly recognized by both academia and industry. It has been almost 15 years since the concept of the DT was initially proposed. To date, many DT applications have been successfully implemented in different industries, including product design, production, prognostics and health management, and some other fields. However, at present, no paper has focused on the review of DT applications in industry. In an effort to understand the development and application of DTs in industry, this paper thoroughly reviews the state-of-the-art of the DT research concerning the key components of DTs, the current development of DTs, and the major DT applications in industry. This paper also outlines the current challenges and some possible directions for future work.

Augmented Reality (AR)-assisted exercises for the enhancement of finger functions have been developed for many years and have proven to be effective. Most applications tracked the motion of upper-extremity using handheld trackers, which are not convenient to put on and take off. Bare-hand tracking algorithms have been utilized to address this limitation; however, only few AR-assisted healthcare approaches have been designed to train upper-extremity skills using bare-hand tracking, and even fewer approaches detect the Range of Motion (ROM) of fingers with low-cost devices. This paper presents a low-cost and multi-modal residential-based AR-assisted therapeutic healthcare exercise system. A computer vision-based bare-hand interaction method is proposed in this system. This method is designed to estimate finger bending degrees through their projection lengths. With an accurate feature detection strategy, this method is able to detect the full ROM using two web cameras. This system also incorporates a vibration wrist band to stimulate the users with tactile feedback, and an assessment module to evaluate user performance.

A new vision-based framework and system for human–computer interaction in an Augmented Reality environment is presented in this article. The system allows the users to interact with computer-generated virtual objects using their hands directly. With an efficient color segmentation algorithm, the system is adaptable to different light conditions and backgrounds. It is also suitable for real-time applications. The dominant features on the palm are detected and tracked to estimate the camera pose. After the camera pose relative to the user's hand has been reconstructed, 3D virtual objects can be augmented naturally onto the palm for the user to inspect and manipulate. With off-the-shelf web camera and computer, natural bare-hand based interactions with 2D and 3D virtual objects can be achieved with low cost.

Repetitive practices can facilitate the rehabilitation of the motor functions of the upper extremity for stroke survivors whose motor functions have been impaired. An affordable rehabilitation system for hand motion would be useful to provide intensive training to the patients. Augmented Reality (AR) can provide an intuitive interface to the users with programmable environment and realism feelings. In this research, a low cost viable rehabilitation system for hand motion is developed based on the AR technology. In this application, seamless interaction between the virtual environment and the real hand is supported with markers and a self-designed data-glove. The data-glove is low cost (<;200 Singapore dollars); it is used to detect the flexing of the fingers and control the movements of the virtual hands.

This paper presents a novel method to determine the multi-dimensional workspace boundary of parallel manipulators using a modified Particle Swarm Optimization (MPSO) framework. A generic objective (fitness) function that gives optimum value at points where a manipulator reaches its limit is formulated. By finding these points using MPSO, the workspace boundary of a parallel manipulator can be constructed. To demonstrate the method, the mapping of the boundary of a Stewart platform workspace is presented. The generic objective function can be modified easily to include any number of parameters and constraints, and this allows the method to be adapted for solving the workspace boundary problem of other parallel manipulators.

Abstract This paper presents an implementation of machining simulation in a real machining environment applying Augmented Reality (AR) technology. Thisin situmachining simulation system allows a machinist to analyze the simulation process, adjust the machining parameters, and observe the results in real‐time in a real machining environment. Such a system is useful for machinists and trainees during the trial and learning stages, allowing them to experiment with different machining parameters on a real machine without having to worry about possibilities of machine and tool breakages. Three research aspects, namely, the tracking and registration module, the physical simulation module, and the implementation and performance of these proposed methodologies are discussed in details in this paper. Experiments were conducted on a real 3‐axis CNC machine to validate and evaluate the performance of the system and the feedback from a survey carried out with the experiments is very positive. Copyright © 2009 John Wiley & Sons, Ltd.

This paper presents a differential GPS carrier phase technique for 3D outdoor position tracking in mobile augmented reality (AR) applications. It has good positioning accuracy, low drift and jitter, and low computation requirement. It eliminates the resolution of integer ambiguities. The position from an initial point is tracked by accumulating the displacement in each time step, which is determined using Differential Single Difference. Preliminary results using low cost GPS receivers show that the position error is 10 cm, and the drift is 0.001 ms-1, which can be compensated using linear models. Stable and accurate augmentations in outdoor scenes are demonstrated.

In this paper, an augmented reality (AR) collaborative design system to support multiple-view product representation in manufacturing is presented. This system can be used to enhance design discussion using the real objects and natural communication behaviors, such as voice inputs and gestures, and also support communication of a geographically dispersed multi-disciplinary team on a network in a concurrent engineering environment. In addition, during the life cycle development of a product, users performing different operations in this life cycle can view different information of the product that is relevant to their tasks through head-mounted devices (HMDs). The information, which is filtered using a client's interest list, i.e. the process information which is of the greatest interest to a client, and deposited in the users' respective databases, can be displayed using computer graphics such as video, text, etc. Furthermore, users can freely make changes to the product model in this AR environment

Machining process simulation and monitoring have been actively researched. However, technologies developed in these two areas have not really been employed in the industries. Human operators are still the vital factor in any manufacturing processes. Augmented reality (AR), a recently developed technology that seamlessly combines virtual information and the real environment, has been applied in various applications. AR technology, as a media between the reality, the operator and the digital world, has not yet been applied in the machining field. This paper discusses the possibilities and feasibilities of this fusion. Machining simulation based on a volumetric model is implemented. The system architecture and some implementation results are presented

Registration is one of the most difficult problems in augmented reality (AR) systems. In this paper, a simple registration method using natural features based on the projective reconstruction technique is proposed. This method consists of two steps: embedding and rendering. Embedding involves specifying four points to build the world coordinate system on which a virtual object will be superimposed. In rendering, the Kanade-Lucas-Tomasi (KLT) feature tracker is used to track the natural feature correspondences in the live video. The natural features that have been tracked are used to estimate the corresponding projective matrix in the image sequence. Next, the projective reconstruction technique is used to transfer the four specified points to compute the registration matrix for augmentation. This paper also proposes a robust method for estimating the projective matrix, where the natural features that have been tracked are normalized (translation and scaling) and used as the input data. The estimated projective matrix will be used as an initial estimate for a nonlinear optimization method that minimizes the actual residual errors based on the Levenberg-Marquardt (LM) minimization method, thus making the results more robust and stable. The proposed registration method has three major advantages: 1) It is simple, as no predefined fiducials or markers are used for registration for either indoor and outdoor AR applications. 2) It is robust, because it remains effective as long as at least six natural features are tracked during the entire augmentation, and the existence of the corresponding projective matrices in the live video is guaranteed. Meanwhile, the robust method to estimate the projective matrix can obtain stable results even when there are some outliers during the tracking process. 3) Virtual objects can still be superimposed on the specified areas, even if some parts of the areas are occluded during the entire process. Some indoor and outdoor experiments have been conducted to validate the performance of this proposed method.

In AR systems, registration is one of the most difficult problems currently limiting their application. In this paper, we propose a simple registration method using projective reconstruction. This method consists of two steps: embedding and tracking. Embedding involves specifying four points to build the world coordinate system on which a virtual object will be superimposed. In tracking, a projective reconstruction technique is used to track these four specified points to compute the model view transformation for augmentation. This method is simple, as only four points need to be specified at the embedding stage and the virtual object can then be easily augmented onto a real scene from a video sequence. In addition, it can be extended to a scenario using the projective matrix that has been obtained from previous registration results using the same AR system. The proposed method has three advantages: 1) It is fast because the linear least square method can be used to estimate the related matrix in the algorithm and it is not necessary to calculate the fundamental matrix in the extended case. 2) A virtual object can still be superimposed on a related area even if some parts of the specified area are occluded during the whole process. 3) This method is robust because it remains effective even when not all the reference points are detected during the whole process, as long as at least six pairs of related reference points correspondences can be found. Some experiments have been conducted to validate the performance of the proposed method.

Abstract In this paper, we propose and develop an easy control tool called Virtual Interaction Panel (VirIP) for Augmented Reality (AR) systems, which can be used to control AR systems. This tool is composed of two parts: the design of the VirIPs and the tracking of an interaction pen using a Restricted Coulomb Energy (RCE) neural network. The VirIP is composed of some virtual buttons, which have meaningful information that can be activated by an interaction pen during the augmentation process. The interaction pen is a general pen‐like object with a certain color distribution. It is tracked using a RCE network in real‐time and used to trigger the VirIPs for AR systems. In our system, only one camera is used for capturing the real world. Therefore, 2D information is used to trigger the virtual buttons to control the AR systems. The proposed method is real‐time because the RCE‐based image segmentation for a small region is fast. It can be used to control AR systems quite easily without any annoying sensors attached to entangling cables. This proposed method has good potential in many AR applications in manufacturing, such as assembly without the need for object recognition, collaborative product design, system control, etc. Copyright © 2004 John Wiley & Sons, Ltd.

Fixture design is an important manufacturing activity which affects the quality of parts produced. In order to develop a viable computer-aided fixturing tool, the fixture-workpiece system has to be accurately modeled and analyzed. This paper describes the modeling, analysis, and verification of optimal fixturing configurations by the methods of force closure, optimization, and finite-element modeling (FEM). Force closure has been employed to find optimal clamping positions and sequencing, while optimization is used for determining the minimum clamping forces required to balance the cutting forces. The developed FEM is able to determine in detail what are the reaction forces, workpiece displacement, deformation in the workpiece and fixtures. In order to produce a more accurate model for predicting the behavior of the fixture-workpiece system, the developed FEM includes fixture stiffness, which past models have assumed as rigid bodies. The reaction forces on the locators are experimentally verified. A sensor-embedded experimental fixturing setup was developed to verify the modeling and the data was used to compare with the FEM. Note to Practitioners-In practice, the machinist applies excessive force to hold the part on machine table, so that it does not move when machining is carried out. However, while machining flexible parts or clamping on finished surfaces, care must be taken so that only necessary and optimal clamping forces are applied, to avoid deforming the part or damaging the surface. Therefore, in this study, we have developed a mathematical model to characterize the mechanics and conditions for a fixture to hold a given part. The developed mathematical model computes the optimal clamping forces required to hold the part in position, given the cutting conditions. The result obtained was verified by conducting the machining test using the developed sensor-embedded experimental fixture setup. The FEM was also developed to predict the behavior of fixture-workpiece system and the results obtained were encouraging.

This paper describes a multi-agent approach to the integration of product design, manufacturability analysis, process planning, and assembly in a distributed manner. The objective is to develop a distributed concurrent engineering system to allow geographically dispersed entities to work cooperatively towards overall system goals. The proposed model considers constraints and requirements from the different product development cycles in the early development phases and fully supports the concept of design-for-manufacturability. This methodology uses conflict resolution (CR) techniques and design-improvement suggestions to refine the initial product design. The model comprises six service agents, a console agent and a facilitator agent. Each service agent is used to model different product development phases, and the console agent acts as an interacting interface between designers and the system, while the facilitator is responsible for the decomposition and dispatch of tasks, and solving conflicts of poor designs. The implementation methodology provides an effective way of enabling heterogeneous legacy software tools in communicating and coordinating with others over the Web through the use of Java-based agent wrapping techniques.

A problem in traditional computer-aided process planning (CAPP) is that the multiple planning tasks are treated in a linear manner. This leads to an over-constrained overall solution space and the final solution is normally far from optimal or even non-feasible. The paper presents a novel CAPP model for machined parts in a job shop environment that contains customer-specified machine tools and cutters. The approach models process planning problems in a concurrent manner to generate the entire solution space by considering the multiple planning tasks, i.e., operations (machine, tool, and tool approach direction) selection and operations sequencing simultaneously. Precedence relationships among all the operations required for a given part are used as the constraints for the solution space. The relationship between an actual sequence and the feasibility of applying an operation is also considered. An algorithm based on simulated annealing (SA) has been developed to find the optimal solution. It is expected that such a planning tool can be used in different job shops for optimal process plan generation.

This paper presents a novel computer-aided process planning model for machined parts to be made in a job shop manufacturing environment. The approach deals with process planning problems in a concurrent manner in generating the entire solution space by considering the multiple decision-making activities, i.e., operation selection, machine selection, setup selection, cutting tool selection, and operations sequencing, simultaneously. Genetic algorithms (GAs) were selected due to their flexible representation scheme. The developed GA is able to achieve a near-optimal process plan through specially designed crossover and mutation operators. Flexible criteria are provided for plan evaluation. This technique was implemented and its performance is illustrated in a case study. A space search method is used for comparison.