Discrete-event simulation is a widely used method for modeling complex systems and evaluating their performance in production. However, the management of discrete-event simulation models and their associated data can be challenging, especially when dealing with large and heterogeneous data sets. In this paper, we present a knowledge graph-based framework that facilitates the management of discrete-event simulation models by representing them and their associated data as a knowledge graph. Our framework enables the integration of multiple data sources, supports the reuse of existing models, and enables the support for different applications based on the knowledge graph. Specifically, we demonstrate the application of our framework to the management of discrete-event simulation models in a use case of automatic model generation for a production line. Our results show that our framework provides an effective solution for managing discrete-event simulation models and their associated data, enabling users to easily query, update, and extend the models while maintaining their integrity and consistency. Overall, our work contributes to the development of knowledge-based solutions for managing complex systems and supports the broader adoption of discrete-event simulation in practice.

Semantic technologies and knowledge graphs are becoming increasingly important in academia and industry. Nevertheless, they remain a rarity in the manufacturing industry, and the advantages they offer are only partially exploited. The same is true for semantic technologies for simulations within production systems. Simulation serves as an essential tool for virtual manufacturing, but the required models are still created by experts in an often elaborate and mostly manual way. In this paper, we derive the requirements for semantic models to capture production simulation knowledge and present an ontological architecture for a knowledge graph for manufacturing and simulation that meets these requirements. The different ontologies of the architecture are based on established production and simulation standards, namely ISA-95 and VDI guidelines. By implementing the ISA-95 standard in an ontology, heterogeneous data from the production system can be integrated. Furthermore, the connection of ISA-95 models to simulation-specific concepts allows the representation of simulation models within the knowledge graph. The implementation of the knowledge graph is shown within a prototypical simulation framework.

Conversational Assistants like Alexa, Siri, and Google Assistant have become part of our everyday lives and faced wide adoption in industry. They support users in various tasks, such as playing songs or booking flights, making users' lives easier. Nevertheless, conversational assistants have not yet found their way into production, even though they would offer advantages, such as intuitive use through natural language and easy access to information. Simulation-based skills can also add a new layer in production by making simulation power accessible in an automated way. In this paper, we aim to discuss the concept of a simulation-based Conversational Assistant by presenting an architecture and the workflow within the architecture.

Recently, ontologies and semantic data technologies have increasingly come back into the focus of research due to the emerging use of knowledge graphs. However, even though the modeling and simulation community has recognized the potential of using this technology for the modeling process, for example for automatic model generation, adaptation or to represent simulation expert knowledge, a general and reusable approach for the aforementioned purposes is still missing. Therefore, in this paper a state of the art review for using knowledge representation during modeling and simulation processes of complex technical systems is conducted, such as factories or process plants with specific focus on the production and logistic domain. Based on that, requirements and benefits of knowledge graphs in this specific domain are evaluated.

In this paper a generic simulation environment is introduced that allows the integration of commercial and research-based simulation tools in a service-oriented manufacturing environment, such as in Industrie 4.0 factories, to support in flexible reconfiguration and scheduling decisions. For this purpose, the simulation environment is connected to a generic middleware and processes parameterized simulation requests via automated model generation and execution, based on current shop floor conditions. Those simulation requests are formulated in a standardized description language for production systems, in this case a derivative of AutomationML (AML) that was extended by simulation specific data sets. Therefore, the Simulation Environment comprises different tool- and non-tool-specific functionalities and components that allow the integration of simulation tools into industrial IT platform environments.

The advances of technologies for mobile robotics enable the application of robots to increasingly complex tasks. Cleaning office buildings on a daily basis is a problem that could be partially automatized with a cleaning robot that assists the cleaning professional yielding a higher cleaning capacity. A typical task in this domain is the selective cleaning, that is a focused cleaning effort to dirty spots, which speeds up the overall cleaning procedure significantly. To enable a robotic cleaner to accomplish this task, it is first necessary to distinguish dirty areas from the clean remainder. This paper discusses a vision-based dirt detection system for mobile cleaning robots that can be applied to any surface and dirt without previous training, that is fast enough to be executed on a mobile robot and which achieves high dirt recognition rates of 90% at an acceptable false positive rate of 45%. The paper also introduces a large database of real scenes which was used for the evaluation and is publicly available.

At the core of every object recognition system lies the development and integration of distinct feature descriptors to create object representations robust against varying perspectives or lightning conditions. Recent work has primarily focused on the development of distinct point features. While these features achieve impressive recognition results, point features fail to capture the shape and appearance of an object with less or even without texture. This paper proposes a novel method for the rapid and dense computation of 2D and 3D image cues from RGB-D data to target the recognition of objects without rich texture and a global histogram-based descriptor for the distinct description of object models.

This paper investigates existing methods for 3D point feature description with a special emphasis on their expressiveness of the local surface geometry. We choose three promising descriptors, namely Radius-Based Surface Descriptor (RSD), Principal Curvatures (PC) and Fast Point Feature Histograms (FPFH), and present an approach for each of them to show how they can be used to classify primitive local surfaces such as cylinders, edges or corners in point clouds. Furthermore these descriptor-classifier combinations have to hold an in-depth evaluation to show their discriminative power and robustness in real world scenarios. Our analysis incorporates detailed accuracy measurements on sparse and noisy point clouds representing typical indoor setups for mobile robot tasks and considers the resource consumption to assure real-time processing.

Robots need a representation of their environment to reason about and to interact with it. Different 3D perception and modeling approaches exist to create such a representation, but they are not yet easily comparable. This work tries to identify best practice algorithms in the domain of 3D perception and modeling with a focus on environment reconstruction for robotic applications. The goal is to have a collection of refactored algorithms that are easily measurable and comparable. The realization follows a methodology consisting of five steps. After a survey of relevant algorithms and libraries, common representations for the core data-types Cartesian point, Cartesian point cloud and triangle mesh are identified for use in harmonized interfaces. Atomic algorithms are encapsulated into four software components: the Octree component, the Iterative Closest Point component, the k-Nearest Neighbors search component and the Delaunay triangulation component. A sample experiment demonstrates how the component structure can be used to deduce best practice.

A growing number of modern computer vision applications like object recognition, collision avoidance and scene understanding demand accurate and dense 3D representations of their environment. To improve existing procedures for 3D data acquisition this paper proposes a novel method for sensor combination on a stereo and a Time-of-Flight camera system. By calibrating the two sensor systems to each other, valid measurements from the 2.5D Time-of-Flight sensor are converted to disparity guesses within the stereo system. The disparity guesses from the Time-of-Flight data constrain the correspondence search results from the stereo matching algorithm. It is empirically shown, that the proposed method effectively enhance the results from stereo vision, especially in structureless areas where stereo correspondence search fails. The method is evaluated on the camera system of the service robot Care-O-bot® 3.

State-of-the-art local feature descriptors like SIFT or SURF require a significant amount of computational power which prevents their usage in applications with real time constraints. Despite recent efforts to simplify the calculation of feature descriptors, a faster computation comes often to the disadvantage of weakening the invariance to rotation or scale. Recently, Tola et al. introduced DAISY, a new local feature descriptor for wide-baseline matching across stereo image pairs. It is shown that DAISY outperforms SIFT in terms of matching accuracy while being computed significantly faster. This paper takes on the idea of DAISY by proposing a rotational invariant extension of the descriptor, called O-DAISY, and outlining its implementation on FPGA to achieve real time performance. The results are benchmarked against its original version and against the widely used descriptors BRIEF and SURF on a standardized image set.

In this paper we propose a new diversity measure based on the correlation of bit strings for the analysis of genetic programming (GP) experiments. The diversity measure has been applied to analyse the impact of pruning on the diversity of a population during genetic programming and its relation to the convergence time of the fitness function. To show the usability of the proposed diversity measure a GP experiment is introduced where simulated Jasmine robots have to learn a collision avoidance behaviour to find their way through a maze. A full analysis of this experiment is given with different fixed pruning strategies in respect to the population diversity and fitness. The GP has been done on behaviour-based robot controllers implemented in MDL2isin. MDL2isin has the advantage that it provides a very compact bit string representation of the control programme, which can be used for diversity analysis.

This paper introduces Care-O-bot®3, a highly integrated and compact service robot with manipulation, navigation and vision capabilities. In particular, Care-O-bot®3 combines the best of available technology including a 7 DOF light-weight arm, an omnidirectional platform and many high-end sensors along with a sustainable, end user oriented design concept enabling many interaction possibilities.

Abstract Common practice in brain research and brain surgery involves the multi‐modal acquisition of brain anatomy and brain activation data. These highly complex three‐dimensional data have to be displayed simultaneously in order to convey spatial relationships. Unique challenges in information and interaction design have to be solved in order to keep the visualization sufficiently complete and uncluttered at the same time. The visualization method presented in this paper addresses these issues by using a hybrid combination of polygonal rendering of brain structures and direct volume rendering of activation data. Advanced rendering techniques including illustrative display styles and ambient occlusion calculations enhance the clarity of the visual output. The presented rendering pipeline produces real‐time frame rates and offers a high degree of configurability. Newly designed interaction and measurement tools are provided, which enable the user to explore the data at large, but also to inspect specific features closely. We demonstrate the system in the context of a cognitive neurosciences dataset. An initial informal evaluation shows that our visualization method is deemed useful for clinical research.

The goal of stylization is to provide an abstracted representation of an image that highlights specific types of visual information. Recent advances in computer graphics techniques have made it possible to render many varieties of stylized imagery efficiently making stylization into a useful technique, not only for artistic, but also for visualization applications. In this paper, we report results from two sets of experiments that aim at characterizing the perceptual impact and effectiveness of three different stylization techniques in the context of dynamic facial expressions. In the first set of experiments, animated facial expressions are stylized using three common techniques (brush, cartoon, and illustrative stylization) and investigated using different experimental measures. Going beyond the usual questionnaire approach, these experiments compare the techniques according to several criteria ranging from subjective preference to task-dependent measures (such as recognizability, intensity) allowing us to compare behavioral and introspective approaches. The second set of experiments use the same stylization techniques on real-world video sequences in order to compare the effect of stylization on natural and artificial stimuli. Our results shed light on how stylization of image contents affects the perception and subjective evaluation of both real and computer-generated facial expressions.

In video see-through augmented reality (AR), virtual objects are overlaid over digital video images. One particular problem of this image mixing process is that the visual appearance of the computer graphics differs strongly from the real background image. The reason for this is that typical AR systems use fast but simple real-time rendering techniques for displaying virtual objects. In this paper, methods for reducing the impact of three effects which make virtual and real objects easily distinguishable are presented. The first effect is camera image noise, which is contained in the data delivered by the image sensor used for capturing the real scene. The second effect considered is edge aliasing, which makes distinguishing virtual objects from real objects simple. Finally, we consider motion blur, which is caused by the temporal integration of color intensities in the image sensor during fast movements of the camera or observed objects. In this paper, we present a system for generating a realistic simulation of image noise based on a new camera calibration step. Additionally, a rendering algorithm is introduced, which performs a smooth blending between the camera image and virtual objects at their boundary in order to reduce aliasing. Lastly, a rendering method is presented, which produces motion blur according to the current camera movement. The implementation of the new rendering techniques utilizes the programmability of modern graphics processing units (GPUs) and delivers real-time frame rates.

Recently, we have proposed a novel approach to generating augmented video streams. The output images are a non-photorealistic reproduction of the augmented environment. Special stylization methods are applied to both the background camera image and the virtual objects. This way, the graphical foreground and the real background images are rendered in a similar style, so that they are less distinguishable from each other. Here, we present a new algorithm for the cartoon-like stylization of augmented reality images, which uses a novel post-processing filter for cartoon-like color segmentation and high-contrast silhouettes. In order to make a fast post-processing of rendered images possible, the programmability of modern graphics hardware is exploited. The system is capable of generating a stylized augmented video stream of high visual quality at real-time frame rates.

Indirect volume rendering is a widespread method for the display of volume datasets. It is based on the extraction of polygonal iso-surfaces from volumetric data, which are then rendered using conventional rasterization methods. Whereas this rendering approach is fast and relatively easy to implement, it cannot easily provide an understandable display of structures occluded by the directly visible iso-surface. Simple approaches like alpha-blending for transparency when drawing the iso-surface often generate a visually complex output, which is difficult to interpret. Moreover, such methods can significantly increase the computational complexity of the rendering process. In this paper, we therefore propose a new approach for the illustrative indirect rendering of volume data in real-time. This algorithm emphasizes the silhouette of objects represented by the iso-surface. Additionally, shading intensities on objects are reproduced with a monochrome hatching technique. Using a specially designed two-pass rendering process, structures behind the front layer of the iso-surface are automatically extracted with a depth peeling method. The shapes of these hidden structures are also displayed as silhouette outlines. As an additional option, the geometry of explicitly specified inner objects can be displayed with constant translucency. Although these inner objects always remain visible, a specific shading and depth attenuation method is used to convey the depth relationships. We describe the implementation of the algorithm, which exploits the programmability of state-of-the-art graphics processing units (GPUs). The algorithm described in this paper does not require any preprocessing of the input data or a manual definition of inner structures. Since the presented method works on iso-surfaces, which are stored as polygonal datasets, it can also be applied to other types of polygonal models.

The ultimate goal of augmented reality is to provide the user with a view of the surroundings enriched by virtual objects. Practically all augmented reality systems rely on standard real-time rendering methods for generating the images of virtual scene elements. Although such conventional computer graphics algorithms are fast, they often fail to produce sufficiently realistic renderings. The use of simple lighting and shading methods, as well as the lack of knowledge about actual lighting conditions in the real surroundings, cause virtual objects to appear artificial. In this paper, we propose an entirely novel approach for generating augmented reality images in video see-through systems. Our method is based on the idea of applying stylization techniques for reducing the visual realism of both the camera image and the virtual graphical objects. A special painterly image filter is applied to the camera video stream. The virtual scene elements are generated using a non-photorealistic rendering method. Since both the camera image and the virtual objects are stylized in a corresponding cartoon-like or sketch-like way, they appear very similar. As a result, the graphical objects seem to be an actual part of the real surroundings. We describe both the new painterly filter for the camera image and the non-photorealistic rendering method for virtual scene elements, which has been adapted for this purpose. Both are fast enough for generating augmented reality images in real-time and are highly customizable. The results obtained using our method, are very promising and show that it improves immersion in augmented reality.

The goal of augmented reality is to provide the user with a view of the surroundings enriched by virtual objects. Practically all augmented reality systems rely on standard real-time rendering methods for displaying graphical objects. Although such conventional computer graphics algorithms are fast, they often fail to produce sufficiently realistic renderings. Therefore, virtual models can easily be distinguished from the real environment. We have recently proposed a novel approach for generating augmented reality images [4]. Our method is based on the idea of applying stylization techniques for adapting the visual realism of both the camera image and the virtual graphical objects. Since both the camera image and the virtual objects are stylized in a corresponding way, they appear very similar. Here, we present a new method for the stylization of augmented reality images. This approach generates a painterly brush stroke rendering. The resulting stylized augmented reality video frames look similar to paintings created in the pointillism style. We describe the implementation of the camera image filter and the non-photorealistic rendererfor virtual objects. These components have been newly designed or adapted for this purpose. They are fast enough for generating augmented reality images in near real-time (more than 14 frames per second).

Utilizing augmented reality for applications in medicine has been a topic of intense research for several years. A number of challenging tasks need to be addressed when designing a medical AR system. These include the import and management of medical datasets and preoperatively created planning data, the registration of the patient with respect to a global coordinate system, and accurate tracking of the camera used in the AR setup as well as the respective surgical instruments. Most research systems rely on specialized hardware or algorithms for realizing augmented reality in medicine. Such base technologies can be expensive or very time-consuming to implement. In this paper, we propose an alternative approach of building a surgical AR system by harnessing existing, commercially available equipment for image guided surgery (IGS). We describe the prototype of an augmented reality application, which receives all necessary information from a device for intraoperative navigation.

The support of surgical interventions has long been in the focus of application-oriented augmented reality research. Modern methods of surgery, like minimally-invasive procedures, can benefit from the additional information visualization provided by augmented reality. The usability of medical augmented reality depends on a rendering scheme for virtual objects designed to generate easily and quickly understandable augmented views. One important factor for providing such an accessible reality augmentation is the correct handling of the occlusion of virtual objects by real scene elements. The usually large volumetric datasets used in medicine are ill-suited for use as phantom models for static occlusion handling. We present a simple and fast preprocessing pipeline for medical volume datasets which extracts their visual hull volume. The resulting, significantly simplified visual hull iso-surface is used for real-time static occlusion handling in our AR system, which is based on off-the-shelf medical equipment.

Segmentation of the tracheo-bronchial tree of the lungs is notoriously difficult. This is due to the fact that the small size of some of the anatomical structures is subject to partial volume effects. Furthermore, the limited intensity contrast between the participating materials (air, blood, and tissue) increases the segmentation of difficulties. In this paper, we propose a hybrid segmentation method which is based on a pipeline of three segmentation stages to extract the lower airways down to the seventh generation of the bronchi. User interaction is limited to the specification of a seed point inside the easily detectable trachea at the upper end of the lower airways. Similarly, the complementary vascular tree of the lungs can be segmented. Furthermore, we modified our virtual endoscopy system to visualize the vascular and airway system of the lungs along with other features, such as lung tumors.