Although AR has a long history in the area of maintenance and service-support in industry, there still is a lack of lightweight, yet practical solutions for handheld AR systems in everyday workflows. Attempts to support complex maintenance tasks with AR still miss reliable tracking techniques, simple ways to be integrated into existing maintenance environments, and practical authoring solutions, which minimize costs for specialized content generation.

Within the German research project ARVIDA a large consortium of industrial Virtual and Augmented Reality users, of technology providing companies and research institutes cooperate on the establishment of highly flexible web-based reference architecture for Augmented Reality applications. The use of web technologies is motivated by modern web standards as WebGL or WebRTC supporting e.g. real time rendering of 3D-content of video streaming within Web-Browsers. Thereby, the use of Web technologies not only offers the possibility to develop applications platform and OS independent but it also facilitates the integration of Augmented Reality into industrial workflows or PDM environments. The developed reference architecture offers RESTful tracking, rendering and interaction services that foster the combination and exchange of different algorithms with the aim to ft the technology to the specific requirements of an AR-applications in an optimal way.

Although AR has a long history in the area of maintenance and service-support in industry, there still is a lack of lightweight, yet practical solutions for handheld AR systems in everyday workflows. Attempts to support complex maintenance tasks with AR still miss reliable tracking techniques, simple ways to be integrated into existing maintenance environments, and practical authoring solutions, which minimize costs for specialized content generation. We present a general, customisable application framework, allowing to employ AR and VR techniques in order to support technicians in their daily tasks. In contrast to other systems, we do not aim to replace existing support systems such as traditional manuals. Instead we integrate well-known AR- and novel presentation techniques with existing instruction media. To this end practical authoring solutions are crucial and hence we present an application development system based on web-standards such as HTML,CSS and X3D.

The success of smart phone technologies changed the way information is processed as more and more geo-referenced structures are emerging linking information to specific locations within our environment. Thereby, Augmented Reality has become a key technology as it analyses the smart phone sensor data (camera, GPS, inertial) to derive the detailed pose of the smart phone, with the aim to correlate our real environment to the geo-referenced information space. In particular, this is relevant for application fields where 3Dmodels are used in planning and organization processes as e.g. facility management. In facility construction and management Building Information Model (BIM) was established as a standard that not only holds the 3D-building-geometry but encompasses pipe/electrical systems as well as semantic building information, for example properties and conditions of building components. With this motivation our work integrates BIM and Augmented Reality.

Within the research project “iStar — Interactive See-Through Augmented-Reality Display” the Fraunhofer Gesellschaft developed Augmented Reality goggles comprising a VGA OLED microdisplay with embedded image sensor aimed on gaze-control and see-through head-mounted optics. The active area of the bi-directional microdisplay consists of nested display and image sensor (embedded camera) pixels surrounded by a second image sensor (frame camera) as well as driving and control circuitry (c.f. Table). The display and image sensor systems are electrically independent of one another, simply interacting via synchronization signals. iStar also includes a developer kit integrating Eye-Tracking software, AR system and application demonstrators. The topic of high-contrast See-Through HMDs still forms an important research topic within the AR community, thereby iStar not only offers a light-weight display solution but it also integrates camera sensors into the display to support Eye-Tracking. With the presented demonstrators the tutorial attendees can evaluate the possibilities and the maturity of the developed technologies. The attendees will get an overview to requirements, solution possibilities and open research topics in the field of microdisplays, optics and software development for the realization of interactive see-Through HMDs. Feedback to learned objectives will be evaluated in questioners.

As large ships are never produced in masses, it often occurs that the construction process and production process overlap in time. Many shipbuilding companies have problems with discrepancies between the construction data and the real built ship. The assembly department often has to modify CAD data for a successful installation. We present an augmented reality system, where a user can visualize the construction data of pipes and modify these in the case of misalignment, collisions or any other conflicts. The modified pipe geometry can be stored and further used as input for CNC pipe bending machines. To guarantee an exactly orthogonal passage of the pipes through aligning bolt holes, we integrated an optical measurement tool into the pipe alignment process.

Augmented Reality (AR) is a technology which has become very popular in the last years. In this context also the idea of using of AR for training applications has become very important. AR offers a large potential for training only if the training is well focused to the skills that have to be trained and if the training protocol is well designed. On the other hand, the generation of the training content to be transferred via AR is a comprehensive problem that is addressed in this paper. Thus, this paper tries to describe the whole chain of implementations and general aspects involved in the creation of AR training applications, including examples for used multimodal devices. This chain starts with the capturing of expert actions to be hold in the "digital representation of skill". The digital representation of skill is transferred to the training protocol that specifies the storyboard of the AR training session. The paper includes two different implementations of AR training systems and describes the general idea of informational abstraction from low level data up to interaction and from design to application.

This paper gives an overview of a multimodal training platform developed for minimally invasive robotic surgery, based on the DLR MiroSurge system. It describes the technological components and integration of the hardware and software platform and presents the first integrated training tasks that enable surgeons to get familiar with the robotic system. The training platform shares the same surgical operator workstation as MiroSurge and simulates the behaviour of the telemanipulator arms and the surgical instruments. Like the real system the training platform provides haptic feedback and 3D-vision. However instead of the real telemanipulator itself, a virtual environment with abstracted tasks is connected to the operator workstation. This allows reduction in costs, to provide various levels of difficulty, and to focus on the skills to be taught. Thus a training platform is presented that aims at training a surgeon's skills in handling the robotic system MiroSurge rather than training surgery in general.

In recent years, the demand for efficient systems which can capture and learn human skills has become increasingly important. In this paper an approach for acquiring and recognizing human assembly skills is presented. The underlying workflows of assembly skills are captured by using a simple multi-sensor data glove and camera tracking. To avoid the processing of redundant information, at first the relevant tasks of a workflow are identified by analyzing measuring information of the multi-sensor capturing system. Thus, only relevant data is comprised in the representation of a workflow. Unlike common approaches a workflow is modeled as entire unit using a continuous hidden Markov model (HMM). The recognition process of input patterns is based on an adaptive threshold model that can identify known workflow patterns and non-meaningful input patterns as well.

This paper presents a simple yet effective approach for classification of human postures by using a time-of-flight camera. We investigate and adopt linear projection techniques such as Locality Preserving Projections (LPP), Linear Discriminant Analysis (LDA) and Principal Component Analysis (PCA), which are more widespread in face recognition and other pattern recognition tasks. We analyze the relations between LPP and LDA and show experimentally that using LPP in a supervised manner effectively yields very similar results as LDA, implying that LPP may be regarded as a generalization of LDA. Features for offline training and online classification are created by adopting common image processing techniques such as background-subtraction and blob detection to the time-of-flight data.

Computer assisted operation planning systems win more and more recognition in the field of surgery. These systems offer new possibilities to prepare an intervention with the goal to shorten the expansive time in the operation room required for the intervention. The safest and most effective surgical approach should be selected. But often, it is difficult to transfer the output of the planning system to the intra-operative situation and so to consider the planning results in the real intervention. At the Fraunhofer Institute for Computer Graphics (IGD) in Darmstadt and the Centre for Advanced Media Technology (CAMTech) in Singapore, methods are developed to bridge the gap between the external planning session and the intra-operative case: augmented reality (AR) techniques are used to overlap preoperative scanned image data as well as results of the planning session to the operation field.

The use of virtual reality (VR) techniques for computer-aided diagnosis is revolutionizing the medical routing in various medical disciplines. Especially in the field of dentistry VR in combination with CAD/CAM technologies offers a high potential for implant planning/design resulting in a higher accuracy and a shorter therapy time period. The geometry of the teeth is recognisable and digitisable using an intraoral camera, the implant models can be designed with specialized CAD systems and the final implants are generated using a CNC machine. In this context the "Virtual Articulator" (VA) completes the digital manufacturing pipeline and offers the possibility to consider patient-specific jaw biomechanics in the implant generation process. The objective of this project is the development of a system for dental occlusion diagnosis. For this important diagnosis in today's dental routine the mechanical articulator is used. The VA simulates and augments the functionalities of the mechanical articulator. Therefore it takes a digital 3-D representation of the jaws as input data, generates a simulation of the jaw movements, and delivers a dynamical visualization of the occlusion points. The high technical demands to the system are given through the high accuracy necessary for dental applications and the large models that have to be handled in the system.