Real-time functional magnetic resonance imaging neurofeedback (rt-fMRI-NF) is a promising noninvasive brain computer interface (BCI) technique for enhancing self-regulation of affective brain states. However, conventional univariate rt-fMRI-NF approaches struggle to discriminate distributed neural patterns underlying distinct emotions. This study implemented an rt-fMRI semantic neurofeedback (rt-fMRI-sNF) paradigm incorporating real-time representational similarity analysis (rt-RSA) to enable navigation among emotional states. Four emotion-specific base patterns were first derived from functional localizer runs and then used as target patterns during neurofeedback. Using an RSA-informed circular semantic map (CSM), participants received real-time visual feedback indicating both the similarity and intensity of their current brain activity relative to target patterns. Participants were instructed to use mental imagery to shift their brain activity toward the specific target pattern and enhance its intensity. Analyses of localizer data revealed overlapping regional activations across emotions and demonstrated that RSA reliably distinguished between emotional states. Group-level mixed-effects modeling of neurofeedback performance indicated significant within-run improvements and higher initial performance in the second run. Together, these results demonstrate the methodological feasibility of an RSA-informed rt-fMRI-NF framework for multivariate brain-state modulation and establish a foundation for future studies examining its transferability and clinical relevance.

The digital twin of humans is a relatively new concept. While many diverse definitions, architectures, and applications exist, a clear picture is missing on what, in fact, makes a human digital twin. Within this context, researchers and industrial use-case owners alike are unaware about the market potential of the-at the moment-rather theoretical construct. In this work, we draw a holistic vision of the human digital twin, and derive the specification of this holistic human digital twin in form of requirements, stakeholders, and users. For each group of users, we define exemplary applications that fall into the six levels of functionality: store, analyze, personalize, predict, control, and optimize. The functionality levels facilitate an abstraction of abilities of the human digital twin. From the manifold applications, we discuss three in detail to showcase the feasibility of the abstraction levels and the analysis of stakeholders and users. Based on the deep discussion, we derive a comprehensive list of requirements on the holistic human digital twin. These considerations shall be used as a guideline for research and industries for the implementation of human digital twins, particularly in context of reusability in multiple target applications.

Modern modelling approaches fail when it comes to understanding rather than pure supervision of human behavior. As humans become more and more integrated into human-to-anything interactions, the understanding of the human as a whole becomes critical. In this paper, we conduct a structured review of the human digital twin to indicate where modern paradigms fail to model the human agent. Particularly, the mechanistic viewpoint limits the usability of human and general digital twins. Instead, we propose a novel way of thinking about models, states, and their relations: Perspectives-Observer-Transparency. The modelling paradigm indicates how transparency - or whiteness - relates to the abilities of an observer, which again allows to model the penetration depth of a system model into the human psyche. The split in between the human's outer and inner states is described with a perspectives model, featuring the introperspective and the exteroperspective. We explore this novel paradigm by employing two recent scenarios from ongoing research and give examples to emphasize specific characteristics of the modelling paradigm.

The increase in complexity of autonomous systems is accompanied by a need of data-driven development and validation strategies. Advances in computer graphics and cloud clusters have opened the way to massive parallel high fidelity simulations to qualitatively address the large number of operational scenarios. However, exploration of all possible scenarios is still prohibitively expensive and outcomes of scenarios are generally unknown apriori. To this end, the authors propose a method based on bayesian optimization to efficiently learn generative models on scenarios that would deliver desired outcomes (e.g. collisions) with high probability. The methodology is integrated in an end-to-end framework, which uses the OpenSCENARIO standard to describe scenarios, and deploys highly configurable digital twins of the scenario participants on a Virtual Test Bed cluster.

Abstract The growing relevance of artificial intelligence (AI) for technical systems offers significant potential for the realization and operation of autonomous systems in complex and potentially unknown environments. However, unlike classical solution approaches, the functionality of an AI system cannot be verified analytically, which is why data-driven approaches such as scenario-based testing are used. With the increasing complexity of the required functionality of the AI-based system, the quantity, and quality of the data needed for development and validation also increase. To meet this demand, data generated synthetically using simulation is increasingly being used. Compared to the acquisition of real-world reference data, simulation offers the major advantage that it can be configured to test specific scenarios of interest. This paper presents an architecture for the systematic generation of virtual test scenarios to establish synthetically generated test data as an integral part of the development and validation process for AI systems. Key aspects of this architecture are the consistent use of digital twins as virtual 1-to-1 replicas and a simulation infrastructure that enables the generation of training and validation data for AI-based systems in appropriate quantity, quality, and time. In particular, this paper focuses on the application of the architecture in the context of two use cases from different application domains.

Although the skills of robot manipulators are becoming technically more complex, the unprecedented cost-effective access to recently unveiled intelligent robots has the potential to unleash as yet unimagined automation capabilities. A key technology behind this opportunity for companies to gain a competitive edge through an informed and intelligent robotized automation is the robotic digital twin (RDT). As such, the RDT will be instrumental in mirroring targeted properties of a physical robot to obtain a digital sibling flexibly harnessed in virtual testbeds to understand, predict, and shape the robot performance. However, these objectives remain challenging to well-established simulators. This is because the architectural and functional capabilities they support are not sufficiently in-line with ever-growing and varying demands for agile and cost-efficient manipulations. As a consequence, robot stakeholders can hardly use RDTs to unlock opportunities and meet needs from prospective markets. This article contributes to addressing this gap. We introduce a novel concept for the development of a RDT that helps create and add value to current and future robotized cyber-physical applications. Hereinafter, referred to as the value-driven RDT (vdRDT), it systematically captures the robot dynamics and purposefully farms data, about which its services reason, to facilitate insight and deliver capabilities as well as benefits to stakeholders. Experiment results show that vdRDTs enlarge the scope of, adapt to, and revitalize robotized applications carried out in different fields.

In many sectors of the economy, modularization and linkage of distributed structures has significantly increased flexibility. Thereby, individual customer demands and changing conditions can be adressed. While this concept is often applied to closed systems whose control is coordinated by central components or can follow defined processes, concepts for the implementation in decentralized and open systems are missing. Such processes can be found in different domains of application, such as logistics, the construction industry or forestry. The use of active asset administration shells for modularized objects enables elements of a system or products to take over parts of the control and coordination. Production systems are enabled to act autonomously. A service infrastructure provides the necessary directory services to provide the asset administration shells with an entry point and to offer suitable routing services. The overall structure is demonstrated and evaluated using a representative example from forestry.

Digital Twins (DTs), an emerging concept from Industry 4.0, are virtual representations of real technical assets. Multi-domain 3D simulation systems can bring DTs to life, even before their physical counterparts are finished. A DT's internal state can be fed from its real twin or generated by simulation. Access to this high-dimensional state of a DT is the key for various analysis and visualization methods presented in this paper. We introduce a generic formalism of state space for DTs and utilize it in an application scenario for automated driving. Throughout this example, methods for state logging and replays, data analysis, and visualization within 3D simulation frameworks are presented. Clear definitions for state variables, vectors, trajectories, and time series help slicing the DTs' state spaces of enormous dimensionality. The presented methodology does not only support the development of intelligent algorithms for autonomous driving, but is also the basis for further use cases of DTs involving optimization, mental models, and decision support systems.

To create an intuitive access to the robotic system at hand, we propose to use its Digital Twin (DT)—a virtual copy of the used robot with all necessary details of the mechatronic system, its construction, function, use and application. To do so, these DTs are embedded into a Virtual Testbed (VTB)—a software framework for cross-system, -discipline, and -application development on a system level—to gain insight into the complex system by having a bidirectional online data stream and interaction between human, DT, and Real Twin (RT). Intuitive user interfaces, the DT, and hardware interfaces are the requirements for this and can be used to fuse reality and virtuality to develop, monitor, and control the robotic system through the DT. Such DTs can then be used in various applications, here focusing on mobile robots for catastrophic scenarios where an effective coupling of human and robot skills is inevitable for a successful mission.

Digital twins represent real objects or subjects with their data, functions, and communication capabilities in the digital world. As nodes within the internet of things, they enable networking and thus the automation of complex value-added chains. The application of simulation techniques brings digital twins to life and makes them experimentable; digital twins become experimentable digital twins (EDTs). Initially, these EDTs communicate with each other purely in the virtual world. The resulting networks of interacting EDTs model different application scenarios and are simulated in virtual testbeds, providing new foundations for comprehensive simulation-based systems engineering. Its focus is on EDTs, which become more detailed with every single application. Thus, complete digital representations of the respective real assets and their behaviors are created successively. The networking of EDTs with real assets leads to hybrid application scenarios in which EDTs are used in combination with real hardware, thus realizing complex control algorithms, innovative user interfaces, or mental models for intelligent systems.

We present a three-level cognitive system in a learning by demonstration context. The system allows for learning and transfer on the sensorimotor level as well as the planning level. The fundamentally different data structures associated with these two levels are connected by an efficient mid-level representation based on so-called “semantic event chains.” We describe details of the representations and quantify the effect of the associated learning procedures for each level under different amounts of noise. Moreover, we demonstrate the performance of the overall system by three demonstrations that have been performed at a project review. The described system has a technical readiness level (TRL) of 4, which in an ongoing follow-up project will be raised to TRL 6.

Digital Twins are a central component in the concept of a modern, Industry 4.0-ready virtual factory. In order to optimize movement paths that emerge during the simulated operation of such a virtual factory, a Digital Twin of a factory has to include a navigation framework that supports movements for humans, as well as autonomous robots. One possible foundation for such a framework are Building Information Models that include details about the buildings' 3D geometry and semantic annotations. In this publication, we propose a framework that allows for effortless inclusion of Building Information Models and factory equipment as well as a plugin system for a variety of spatial representations (e.g. regular grids or navigation meshes).

Although SMEs would benefit from robotic solutions in assembly, the required invests and efforts for their implementation are often too risky and costly for them. Here, the Horizon 2020 project “ReconCell” aims at developing a new type of highy-reconfigurable multi-robot assembly cell which adresses the particular needs of SMEs. At the Institute for Man- Machine Interaction (MMI), we are developing 3D simulation-based user interfaces for ReconCell as the central technology to enable the fast, easy and safe programming of the system. ReconCell heavily builds on previous developments that are transferred from research and prepared for industrial partners with real use cases and demands. Thus, in this contribution, we describe MMI's software platform that will be the basis of the desired user interfaces for robot simulation and control, assembly simulation and execution, Visual Programming and sensor simulation.

One of the next steps in factory automation might not exclusively be an automation issue, but instead combining robot and human skills to further improve industrial work processes. For various reasons, there is still a low dissemination of hybrid work processes characterized by direct human robot interaction. For instance, it is very difficult to decide which manual work processes are eligible for a transformation to a human-robot interactive process. Thus, the research project MANUSERV delivers a tool to support this decision process. Here, the central concept is a task planning system capable of generating automated as well as hybrid human-robot solutions. Therefore, a structured description of manual work processes forms the input to the planning system. Subsequently, a simulation system verifies and evaluates the proposed solutions and generates the necessary information for a transformation of the planning results to a real application scenario.

The increasingly pervasive usage of robotic automation not only calls for a high performance of physical robot manipulators, but also implies challenging requirements on 3D robot simulators. A skillful reaction of simulated robots to unpredicted contact forces has become an especially stringent necessity to support the development of emerging applications for which a compliant robotic manipulation is indispensable. This paper focuses on the simulation of physics-based virtual testbeds in which simulated robots are made capable of accommodating external forces exerted by the environment on their manipulators. For this, we integrate methods from multi-body dynamics modeling, advanced robotics, and virtual reality to develop a robotics simulation basis that facilitates the employment of compliant robotic manipulations in simulated applications. The validation of this basis in joint space indicates that a physics-based simulation of robots endowed with active physical interaction control can capture and reflect significant effects which characterize the interactive behavior of physical compliant robots.We demonstrate the practical utility of our approach to unlock competitive advantages in fully simulated applications prone to position uncertainty. Furthermore, we show the potential of using compliant digital twins of physical robots to introduce compliant robotic manipulations into real applications.

Today, laser manufacturers investigate and invest in automated assembly approaches in order to meet the high demands on flexibility, autonomy and efficiency and to counter competitive pressure. In the project “DeLas”, a consortium of manufacturers of micro-optical components and manufacturers of assembly equipment for micro-optical components came together with researchers in automation and robotics for transferring state-of-the-art methods in their academic fields to industrial assembly scenarios. Due to a surprising lack of automation and accompanying model- and simulation-based toolchains in this industrial sector so far, automation in micro-optical assembly strongly and newly benefits from tools well-established in other sectors, such as 3d simulation, visual programming and motion planning.

Robot manipulators are increasingly employed in dynamic and uncertain environments. Therefore, weak and strong physical interactions with the surrounding operational environment can occur. Managing the complexity related to this trend requires simulation capabilities going far beyond traditional off-line motion planning of robots in free-space. This paper presents a dynamic simulation that addresses this issue. The simulator conveys insight into the accommodation of external forces during physical interaction, since the simulated robots are endowed with force-sensitive and compliance control capabilities. Further, a symbiotic simulation of compliance control enables physical interaction between a human operator and a real position-controlled robot. Our approach builds upon a comprehensive, holistic and versatile coupling of modeling paradigms. It leverages the significance of simulation outcomes by enriching the simulator with realistic robot control techniques. It deepens the understanding of ongoing processes by allowing an overall and customizable view on the robot down to actuators. The results highlight the effectiveness of the eRobotics methodology, which represents the foundation this work is built upon.

We describe a system allowing a robot to learn goal-directed manipulation sequences such as steps of an assembly task. Learning is based on a free mix of exploration and instruction by an external teacher, and may be active in the sense that the system tests actions to maximize learning progress and asks the teacher if needed. The main component is a symbolic planning engine that operates on learned rules, defined by actions and their pre- and postconditions. Learned by model-based reinforcement learning, rules are immediately available for planning. Thus, there are no distinct learning and application phases. We show how dynamic plans, replanned after every action if necessary, can be used for automatic execution of manipulation sequences, for monitoring of observed manipulation sequences, or a mix of the two, all while extending and refining the rule base on the fly. Quantitative results indicate fast convergence using few training examples, and highly effective teacher intervention at early stages of learning.

The continuously widening range of applications of robot manipulators does not only give rise to high performance requirements on robots. It notably implies challenging demands set on robot simulators. These include modeling flexibility and the support of advanced control capabilities, especially in the case of mechanical interaction. This paper focuses on the simulation of compliant robots that can skillfully interact with their environments. To this end, robots in the simulator are endowed with the capability to accommodate external forces. The practical usefulness of the simulator includes the flexible and insightful planning, assessment and enhancement of various interaction scenarios through 3D simulation as well as physical interaction with real robots by using simulated compliant joint trajectories. The approach and results highlight the benefits that can be realized through the synergy between intelligent robotics and virtual reality techniques, as pursued by eRobotics, which strives to provide a comprehensive and versatile software environment for addressing robotics-related issues.

eRobotics is a newly evolving branch of eSystems engineering, providing tools to support the whole life cycle of robotic applications by means of electronic media. To this end, it extends and strengthens the Virtual Testbed idea from space robotics to provide a holistic 3D simulation platform on which complex issues in robotics and automation can be addressed and the most efficient solutions elaborated. In this line of idea, this work introduces a new and systematical eRobotics approach to support process optimization and decision making within a comprehensive 3D simulation environment. We illustrate the importance and performance of this 3D simulation-based analysis and optimization through diverse examples, ranging from planetary exploration and orbital servicing to industrial manufacturing.

Robotic applications are complex, require hardware prototypes and thus are usually a costly endeavor. The research field of eRobotics has been established recently in order to help to cope with the inherent complexity, facilitate the development and significantly cut down the cost of advanced robotics and mechatronics development. The objective is to effectively use electronic media - hence the "e" at the beginning of the term - to achieve the best possible advancements in the development of the robots respective fields of use. The aim of developments in eRobotics is to provide a comprehensive software environment to address the robotics-related issues for the next generation robotic applications.

In recent years, virtual reality has emerged as a key technology for improving and streamlining design, programming, manufacturing and training processes. Based on experiences in the fields of space robotics, industrial manufacturing, multi-physics and virtual prototyping, "virtual testbeds" are currently being designed and implemented. Experience now shows that virtual testbeds also have a high potential to serve as the foundation for e-Robotics as robotics developed to become a computationally intensive science that is best carried out in a collaborative environment and preferably in a highly distributed network environment. So far, the robotics community does not appear to already appreciate the advantages of principles of e-Science being applied to robotics, probably because most robotics applications so far are being built bottom up and are still mostly dedicated to specific problems, e.g. in the domain of controller development. But when it comes to building larger applications like e.g. a space robot mission, planetary exploration rovers, au-tonomous working machines, advanced vehicle assistant systems etc., the demand for highly realistic and collaborative simulation environment increases. The developments of "virtual testbeds" to be presented in this paper describe the symbiotic integration of advanced simulation techniques with distributed computing ca-pabilities and advanced man-machine interaction metaphors to pave the way for e-Robotics with applications not only in space but also in factories, on construction sites - and in the woods.

This paper describes how structures for the flow control in robot programming languages can be translated into Petri nets. Structures common to many robot programming languages are identified and their representation in the realm of Petri nets is described. This common representation of robot programs allows us to apply to them techniques originally developed for Petri nets. An example of how to combine the single structures into Petri nets representing larger robot programs and future prospects of these efforts conclude the work.

It contradicts the human's expectations when humanoid robots move awkwardly during manipulation tasks. The unnatural motion may be caused by awkward start or goal configurations or by probabilistic path planning processes that are often used. This paper shows that the choice of an arm's target configuration strongly effects planning time and how human-like a planned path appears. Human-like goal configurations are found using a criterion from ergonomics research. The knowledge which pose of the Tool Center Point (TCP) can be reached in a natural manner is encapsulated in a restricted reachability map for the robot arm.

The goal of this study is the discrimination of seven tree species. As a well known approach the k-nearest neighbor classifier is compared to a support vector machine based decision tree. This classifier uses advanced support vector machines to implement a hierarchical classification scheme by combining it with decision tree induction. At each node of the decision tree a support vector machine is trained. Furthermore the impact of LIDAR differential data and kernel choice is evaluated. The effects of two separability measures and three grouping strategies in the decision tree induction on the classification results of the support vector machine based decision tree (SVMDT) are studied.

The most promising approach for the design of powerful and complex automation systems is to build a simulation model and simulate it beforehand. Our approach includes the simulation of malfunctions into this development process. Each object and electrical connection in our simulation model has a set of possible malfunctions. A mathematical model describes the impact of malfunctions. We have integrated our malfunction modeling and simulation concepts into an existing simulation system and show applications of our approach for the training of students and maintenance staff. Our approach tackles the problem of training maintenance personnel efficiently. During the built-up of the real automation system, the maintenance staff can gain knowledge of all possible malfunctions using the simulation system.

In recent years, virtual reality has emerged as a key technology for improving and streamlining design, programming, manufacturing and training processes. Based on experiences in the fields of space robotics, industrial manufacturing, multi-physics and virtual prototyping, "virtual testbeds" are currently being designed and implemented. Building on experiences gained in space robotics applications, the idea of "virtual testbeds" currently conquers new fields of applications in the manufacturing industry, on construction sites and even "in the woods". Interestingly, all fields can be supported with a rather generic and common basis so that the different applications can be realized very cost-efficiently. This paper is an attempt to give an overview over different current applications and setups. Furthermore, it will focus on the phases of a product's lifecycle that can already today be efficiently supported by virtual and augmented reality technologies. From a system theoretic standpoint, virtual testbeds are currently bridging the gap between virtual reality and robotics because on the one hand, robotics know how related to kinematics and physics based modeling is being used in virtual worlds to make them "behave realistically", on the other hand, those virtual worlds make very efficient representations of "real worlds" in which planning systems for robotics can try out action alternatives quickly and without danger to the robotic hardware.

This paper describes a novel method for motion generation and reactive collision avoidance. The algorithm performs arbitrary desired velocity profiles in absence of external disturbances and reacts if virtual or physical contact is made in a unified fashion with a clear physically interpretable behavior. The method uses physical analogies for defining attractor dynamics in order to generate smooth paths even in presence of virtual and physical objects. The proposed algorithm can, due to its low complexity, run in the inner most control loop of the robot, which is absolutely crucial for safe Human Robot Interaction. The method is thought as the locally reactive real-time motion generator connecting control, collision detection and reaction, and global path planning.

Simulation in the context of engineering often focuses on very special details of global systems. Robot designers usually begin with the analysis of new actuators and joint designs. This corresponds to a “bottom-up”-strategy in the development of simulation models. For classical fields of application of robotics, e.g. in production plants with a well defined environment this is the approved method, because it allows very detailed insights into the analyzed subsystems. On the other hand, unpredictable effects of the interaction of multiple subsystems may easily be overseen. In particular, nontechnical environments like in moon exploration tasks or in a biological environment like in forestry applications are hard to describe in an analytical way to integrate them into an analytical simulation model. This is why this paper presents the idea and some practical aspects of the development of “Virtual Testbeds”. In a Virtual Testbed, the entire system is simulated as a whole in Virtual Reality - not only small subsystems of a global system. According to the requirements different subsystems are simulated with different levels of detail. In contrast to the classical “bottom-up”-strategy this can be seen as a “top-down”-approach. Therefore the employment of a multi-body dynamics system as a platform for the development of versatile simulation and testing environments is proposed. Using the examples of the evaluation and testing of an extraterrestrial walking exploration robot design and the development of a method for self-localization in forestry, the idea is further deepened. As a special field of attention the integration of a method of soil simulation as a particular requirement of a Virtual Testbed for walking exploration robots is presented.

After giving a rich data basis of our impact tests with standardized crash-test dummies in Part I of this work we address in Part II various aspects related to these tests in a case based discussion. The presented facts, the knowledge gained from our previous work, and the data from Part I lead us to recommendations for standardized crash-testing procedures in robotics. The proposed impact procedures will help to compare blunt robot-human impacts on a common basis. We will discuss additional requirements which will enhance the completeness of testing procedures.

After analyzing fundamental impact characteristics of robot-human collisions in our previous work, the intention in the present paper is to augment existing knowledge in this field, verify previously given statements with standardized equipment of the German Automobile Club (ADAC), and provide a crash-test report for robots in general. Various new insights are achieved and a systematic and extensive set of data is provided. The presented work is divided into two papers. The main purpose of Part I is to give, similarly to reports known from the automobile world1, a fact based and result oriented view on our newest robot crash-test experiments. In Part II detailed discussions of the results listed in the present paper and recommendations towards a standard crash-test protocol for robot safety are carried out.

In many cases, visual tracking is based on detecting, describing, and then matching local features. A variety of algorithms for these steps have been proposed and used in tracking systems, leading to an increased need for independent comparisons. However, existing evaluations are geared towards object recognition and image retrieval, and their results have limited validity for real-time visual tracking. We present a setup for evaluation of detectors and descriptors which is geared towards visual tracking in terms of testbed, candidate algorithms and performance criteria. Most notably, our testbed consists of video streams with several thousand frames naturally affected by noise and motion blur.

This paper will introduce the generic concept of a multi-agent based crowd simulation prototype. The prototype consists of many distinct components that contribute to the system as a whole. Based on the criteria for a human agent model, the agent's module-based, layered architecture is introduced. Subsequently, a closer examination of each module reveals a detailed insight into the agent's architecture. The examined modules are: The agent's finite state machine, its steering behavior, its locomotion model, the path planner and the messaging capability. Finally, with respect to the simulation's general architecture, the optimization techniques to further improve the simulation's runtime performance are discussed. This is especially important, since the simulation should run at interactive frame rates and allow the simulation of as many agents as possible for scalability reasons. The investigated optimization techniques are multi-threading and cell space partitioning. The simulation is implemented in our 3D simulation system VEROSIM, which also handles 3D graphics to visualize the results in real-time.

This paper describes a concept for a task scheduling mechanism in factory simulation systems for tasks with differing cycle times. The goal is to partition tasks in a way that makes parallel computation possible, while maintaining a deterministic behavior.

In 2004, the European COLUMBUS Module is to be attached to the International Space Station. On the way to the successful planning, deployment and operation of the module, computer generated and animated models are being used to optimize performance. Under contract of the German Space Agency DLR, it has become IRF's task to provide a Projective Virtual Reality System to provide a virtual world built after the planned layout of the COLUMBUS module enabling astronauts and scientists to practice operational procedures and the handling of experiments. The possibility for distributed multiuser access to the virtual lab and the visualization of real-world experiment data comprise the key features of the system. Through the ability to share the virtual world, cooperative operations can be practiced easily and trainers and trainees can work together more effectively in the shared virtual environment. The ability to visualize real-world data will be used to introduce measured experimental data into the virtual world online in order to allow realistic interaction with the science reference model hardware: The user's actions in the virtual world are translated into corresponding changes of the inputs of the science reference model hardware; the measured data is then in turn fed back into the virtual world. In order to provide astronauts and scientists with an even more complete insight into various aspects of COLUMBUS and the ISS, the simulation of the COLUMBUS module is currently being extended to include the entire International Space Station. This paper describes the background and the developed features which allow the creation of a virtual world that provides users with the feel of being on board the ISS.

This paper proposes the basic ideas of a proven dynamic collision avoidance approach for multirobot manipulator systems. The flexible use of multirobot systems requires new methodologies to ensure safe and collision-free movement of the controlled robots. The new method CARE (collision avoidance in real-time environments) that is presented in this paper is such a new safety methodology. Comparing this to existing approaches to collision avoidance, this new methodology has substantial advantages: (1) CARE makes sure, that the robots can generate the calculated evasive acceleration and avoid the collision at any time, (2) CARE is based on an intuitive, physically comprehensible and dynamic model of the collision process, which raises the confidence and the acceptance of such a safety system significantly, (3) CARE takes care of all potential collision dangers at the same time and, (4) CARE is model based so that it can be adapted to new robot kinematics or new robot work-cells only by modeling the new environment. All collision avoidance experiments with ERA have been carried out very successfully, whereby CARE has taken full control over the space robot.

Today's robot applications often include multi-robot multi-user environments which are planed thoroughly in 3D before installation. The key to gain the maximum benefit out of the already available models it is to reuse them for other purposes than only robot-simulation. One of these aspects surely is the training of future operators of the cell which is important especially for environments that are difficult to access for training. For example a production line in the automotive industry may not be shut down for a day of emergency-training. In this paper we will show how to reuse already modelled robot cells and mechanisms of the Cosimir simulation-system in emergency-training. We present an approach for graphical visualisation and simulation of fire. We show how to get realistic impressions of fire using advanced particle-simulation and how to use the advantages of particles to trigger states in a modified cellular automata used for the simulation of fire-behaviour. By this we can have a simulated fire developing and reacting on other fires and different substances as water, CO/sub 2/ or oxygen. It turns out that the mechanisms implemented for fire-simulation are also valuable for robot-simulation. So there is a mutual benefit between the robot-simulation and the simulation of fire. The methods proposed in this paper have been implemented and successfully tested on Cosimir, a commercial robot-and VR-simulation-system.

Building on existent robotics knowledge to model and simulate anthropomorphic kinematics is an appealing approach, because sound knowledge gained in the fields of multi-robot and multi-agent systems can be applied. This provides-with little additional effort-the "new human" with capabilities ranging from natural arm movement up to the coordinated operation of his arms and the cooperation between multiple anthropomorphic kinematics. The same general hierarchical control structure that has successfully been used to control multi-robot systems for space and industrial application has therefore been enhanced to incorporate the newly required capabilities. The enhancements focus on a new approach to on the one hand consider human extremities as articulated robots which are mechanically connected to make up the body-and on the other hand to provide a control strategy to move the full body correctly under equilibrium conditions. We provide an overall control structure that preserves the capabilities of the single robots and introduces "sensible couplings" to move the body as a whole in a naturally looking way. As described in the paper, this work is currently being applied to several application fields in industry up to the simulation of astronauts' work on the International Space Station.

The commanding and supervision of complex automation systems for space as well as for terrestrial automation applications is a demanding task. Modern developments in the field of virtual reality (VR) based man machine interfaces have the potential to facilitate such tasks enormously. At the Institute of Robotics Research (IRF) in Dortmund, Germany a variety of practical applications regarding the control of robots over long distances by means of virtual reality based man machine interfaces have been developed and successfully tested. In April 1999, the experiences made were successfully applied to the commanding and supervision of the robot ERA on board the Japanese satellite ETS-VII. In a joint mission between the Japanese Space Agency NASDA, the German Space Agency DLR and IRF, the robot was successfully commanded and supervised by means of "projective virtual reality" methods. The paper describes the key ideas and features of projective virtual reality as well as the metaphors used to augment the virtual world and to make the operation of the robot very intuitive. Before being able to apply the framework of projective virtual reality to this application, two major issues had to be addressed. First, it has to be made sure that the commanded robot can never damage itself or his environment this was addressed by automatic collision avoidance strategies. Secondly, the virtual model has to generated and then to be calibrated-this was addressed by the "TV-view into reality"-metaphor and modern 3D vision algorithms to provide the required online world-model updates.

This paper presents a new approach to the online planning of collision-free robot motions and evasive actions for redundant multi-robot/multi-obstacle environments. The approach, based on the collision avoidance methodology CARE (collision avoidance in real-time environments) presented by Rossmann (1993), is a local planning approach which considers the shortest distance and relative velocity between objects. This results in an intuitively comprehensible model to determine the potential collision and provides the basis of a new way of treating collision free path-planning as an optimization problem. This optimization problem is solved in real-time and provides a mathematically exact solution of the path-planning problem by considering all the static and dynamic obstacles in the environment.

This paper introduces the use of supervisory control techniques for the realization of a new kind of intuitive man machine interfaces for complex automation systems. Such comprehensive user interfaces combine for the first time intuitive commanding capabilities: clear and vivid visualization of the system state as well as interactive training environments especially for the commanding of autonomous robotic systems over long distances. To achieve this, a state oriented modeling technique has been developed. It serves as an object oriented supervisory control framework and was integrated in the IRFs VR-system COSIMIR/sup (R)/ VR.

Experiences in space robotics show, that the user normally has to cope with a huge amount of data. So, only robot and mission specialists are able to control the robot arm directly in teleoperation mode. By means of an intelligent robot control in cooperation with virtual reality methods, it is possible for non-robot specialists to generate tasks for a robot or an automation component intuitively. Furthermore, the intelligent robot control improves the safety of the entire system. The on-ground robot control and command station for the robot arm ERA onboard the satellite ETS-VII builds on a new resource-based action planning approach to manage robot manipulators and other automation components. In the case of ERA, the action planning system also takes care of the "real" robot onboard the satellite and the "virtual" robot in the simulation system. By means of the simulation system, the user can plan tasks ahead as well as analyze and visualize different strategies. The paper describes the mechanism of resource-based action planning, its application to different work cells, the practical experiences gained from the implementation for the on-ground robot control and command station for the robot arm ERA developed in the GETEX project as well as the services it provides to support VR-based man machine interfaces.

The new paradigm of projective virtual reality is presented as a sophisticated new approach to the control and supervision of robotic and automation systems. The idea behind projective virtual reality is to "project" actions that are carried out by users in the virtual world into the real world, primarily by means of robots but also by other means of automation. Robot control technology thus provides the user in the virtual reality with a "prolonged arm" into the physical environment, paving the way for a new quality of a user friendly man machine interface for automation applications. The paper provides the background of projective virtual reality which is based on latest results in the fields of task planning, world modeling, cooperative robot control and sensor-based robot control.

Practical experiences in the development of modern robot control systems showed that the most successful components developed were those that were designed to work in comprehensive automation systems. These components were forced to be implemented in a greater framework and thus to adhere to standards and to standardized interfaces in order to facilitate integration and test of the final system. Although it was suspected in the beginning that system constraints in the form of architectural design guides imposed on the different parts of the realization might hamper the creativity of the developers, it was found that just the opposite was the case. After a training phase the developers managed to "think globally, but act locally", so that e.g. the development, the integration and the tests of a multirobot system for space-applications, a robot-simulation system, a flexible assembly-workcell and a new virtual-reality-system could be conducted in a coherent manner in a very short time.

The aim of the development of modern robot control systems at the Institute of Robotics Research is to provide higher flexibility and autonomy for robot systems, together with consistent and user-transparent concepts for installation, programming and operation of robots in their working environment. An important feature of a flexible future-oriented robot controller is the capability to incorporate information from different classes of sensors and to have standardized communication interfaces with other factory-automation components. To ease the robot systems' operation a graphical user-interface is provided. Furthermore, implicit programming and action planning methods allow robot programming on a much higher level of abstraction than today's procedural programming languages. For industrial applications, this implies easier and quicker adaptation of robot work cells to new tasks and a significant increase of the robots' flexibility to allow the feasibility of small quantity orders. This paper gives a basic description of "the principle of the hierarchical coordinator", the basic system concept for the development of complex control systems at the IRF. This concept has been developed and employed for the design of intelligent, autonomous robot systems, which are described in detail. The applications described range from the design of a flexible assembly work cell to a multirobot-system for autonomous experiment-servicing in a space laboratory module.>

In this paper a new approach to realistic simulation of robotic workcells in industrial manufacturing processes is presented, in which the workcell is composed of different classes of objects like robots, AGVs, conveyor-belts and sensors. The presented simulation system is directed to the areas of offline-programming and workcell layout. The novel approach described includes the embedding of an industrial robot controller into the simulation system by means of a versatile message-based interface and the use of a sophisticated environment model for intuitive simulation of grip and release operations, conveyor-belts and sensors. This is combined with a graphical 3D-visualization of the workcell in real-time and a windows-oriented interface based upon widely spread GUIs on PCs and UNIX-workstations.>