This paper introduces a novel physical annotation system that is designed to generate training data for automated optical inspection. The system uses pointer-based, in-situ interaction to transfer the valuable expertise of trained inspection personnel directly into a machine learning training pipeline. Unlike conventional screen-based annotation methods, our system allows annotation directly on the physical object, providing a more intuitive and efficient way to label data. The core technology uses calibrated, tracked pointers to accurately record user input and convert these spatial interactions into standardised annotation formats compatible with open-source software. A simple projector-based interface also projects visual guidance onto the object to assist users during the annotation process, ensuring greater accuracy and consistency. The proposed concept bridges the gap between human expertise and automated data generation. It enables non-IT experts to contribute to the ML training pipeline. Preliminary evaluation results confirm the feasibility of capturing detailed annotation trajectories and demonstrate that integration with CVAT streamlines the workflow for subsequent ML tasks. This paper details the system architecture, calibration procedures, and interface design, and discusses the concept’s potential contribution to future ML-based automated optical inspection.

We introduce IndEgo, a multimodal egocentric and exocentric dataset addressing common industrial tasks, including assembly/disassembly, logistics and organisation, inspection and repair, woodworking, and others. The dataset contains 3,460 egocentric recordings (approximately 197 hours), along with 1,092 exocentric recordings (approximately 97 hours). A key focus of the dataset is collaborative work, where two workers jointly perform cognitively and physically intensive tasks. The egocentric recordings include rich multimodal data and added context via eye gaze, narration, sound, motion, and others. We provide detailed annotations (actions, summaries, mistake annotations, narrations), metadata, processed outputs (eye gaze, hand pose, semi-dense point cloud), and benchmarks on procedural and non-procedural task understanding, Mistake Detection, and reasoning-based Question Answering. Baseline evaluations for Mistake Detection, Question Answering and collaborative task understanding show that the dataset presents a challenge for the state-of-the-art multimodal models. Our dataset is available at: https://huggingface.co/datasets/FraunhoferIPK/IndEgo

Abstract The field of visual Industrial Anomaly Detection (IAD) has brought forth many new semi-supervised learning methods in recent years. At the same time, there have been few new datasets for benchmarking the methods. The most popular dataset is MVTec-AD dataset, because of its diversity of categories and availability of industrial objects. But many methods already achieve AUROC scores of more than 99 % on the MVTec-AD dataset. The defects of the categories that the dataset provides appear to be easily detectable. Furthermore, there is no existing approach to statistically describe the defects that need to be found in IAD datasets. This paper presents a new dataset for visual industrial anomaly detection and a novel approach for Anomaly Detection Dataset Difficulty assessment with the AD3 score. The new dataset named VIADUCT contains 49 categories and 10,986 high resolution images from eleven different sectors. Through the support of several manufacturing companies, numerous real inspection problems are presented through the dataset. It contains a large number of different defects with detailed pixel-wise annotations. The VIADUCT dataset is compared with other state of the art datasets to underline its added value. Therefore, we provide an overview for each dataset regarding the number of categories, images, defect categories and defects. In addition to these obvious comparisons the defects of the datasets are described with the AD3 score. This novel score is used to analyze the size of the defects and the similarity between the defect and its corresponding object. Using seven selected methods from industrial anomaly detection, a benchmark is performed on the new dataset, showing that there is still potential for improvement. It is shown that the VIADUCT dataset is the largest dataset in the field of image-based industrial anomaly detection. In addition to its very small defects which are hard to recognize, the dataset also offers the greatest variance of possible defects and the most defect classes. Describing the datasets with AD3 score it can be found that VIADUCT dataset have the most inconspicuous defects. With the AD3 score we are able to create a-priori knowledge for every single defect in IAD datasets. The AD3 score correlates with the results of the IAD method benchmark, showing that it can be used to estimate defect detection difficulty. In the future, new objects can be assessed to see whether defects can be recognized using IAD methods before an energy-intensive benchmark is performed. The simple calculation of the AD3 score generates valuable a-priori knowledge and can save resources.

Automating machine tending for high mix - low volume environment provides additional engineering challenges. Current machine tending systems are usually fixed in place and preprogrammed by a human operator. Available mobile robots are limited in their machine tending capabilities as they cannot provide the required accuracy that is necessary for the precise insertion of work pieces into a fixture. This paper presents a mobile machine tending system that uses computer vision and ROS2 to address these limitations and to enable autonomous machine tending. After a brief introduction to the system architecture, the article provides first insights into the systems capabilities based on experimental evaluation. The first experiment addresses the accuracy of the end effector, showing reasonable improvements using low cost sensors. The second experiment evaluates the network load of the distributed control system and discusses the requirements for the operation of a fleet of such robots.

Anomaly detection is one of the most popular fields for computer vision in industrial applications. The idea of training machine learning only on defect-free objects saves enormous amounts of integration effort. The state of the art shows that current methods on public data sets (e.g. MVTec AD data set [1]) have already solved the problem with AUROC segmentation scores of more than 99%. In real-world applications training data is not as ”clean” as in public data sets. This work investigates the changes in detection performance when outliers end up in the training data. For this purpose, the training data is enriched step by step with images of defective objects. The AUROC score and the anomaly score is used as a quality criterion for performance measurement. We show that state of the art methods can be very robust, but that in some scenarios a draw down of 15 percentage points is possible.

Anomaly detection is one of the most popular fields for computer vision in industrial applications. The idea of training machine learning only on defect-free objects saves enormous amounts of integration effort. The state of the art shows that current methods on public data sets (e.g. MVTec AD data set) have already solved the problem with AUROC segmentations scores of more than 99%. But how accurate are these methods really? In this paper, one current method from the field of supervised learning and anomaly detection is evaluated on two problems. Each problem contains a defect pattern that grows in 11 steps. This work shows that the defect is already reliably detected from a relative size of 0.03% of the pixels in the image.

The manufacturing sector has witnessed a rapid rise in the importance of energy-efficient operation. For finding optimal set-points for industrial facilities, optimization problems of increasing complexity occur. Key challenges are the leak of derivative information and the curse of dimensionality. For systematic reduction of the search-space by decomposition of the model, a methodology for the inclusion of topology knowledge in the optimization procedure is developed. An implementation of OptTopo (Optimization based on Topology), embedded in a testbed, demonstrates its advantages compared to popular out-of-the-box-optimization. OptTopo could be integrated in energy management software offering advanced set-point control for complex facilities.

Tool wear is a significant cost factor in metal cutting operations. Former research shows that up to 50% of the expected tool life are wasted. To increase the utilized tool life without risking damages on the work piece, information about the state of the tool wear is necessary and therefore, the wear needs to be measured. Manual measurement is time consuming and expensive, so automatic tool wear measurement is approached by research in many ways, amongst which are computer vision and machine learning. The goal of this paper was the development of a method for automated tool wear classification, identification and quantification. Both computer vision and machine learning were applied to a sample set of 4 types of metal cutting inserts used in tools for super-alloy machining of aircraft engine parts. Tool wear classification yielded accuracies between 84.5 % and 100 %. Flank wear contours are identified optically correct on all investigated insert types, and quantification shows errors of 6-7 % and just 3-4 % of outliers with errors of more than 20 % on the first of the investigated insert types for maximum and average wear land width. Thereby, this paper contributes to a foundation of industrially applicable methods for automated tool wear identification, classification and quantification.

Motion datasets in industrial environments are essential for the research on human-robot interaction and new exoskeleton control. Currently, a lot of Activities of Daily Living (ADL) datasets are available for researchers, but only a few target an industrial context. This paper presents a dataset for a semi-industrial Overhead Car Assembly (OCA) task consisting of synchronized video and 9-Degrees of Freedom (DOF) Inertial Measurement Unit (IMU) data. The dataset was recorded with a soft-robotic exoskeleton equipped with 4 IMUs covering the upper body. It has a minimum sampling rate of 20 Hz, lasts approximately 360 minutes and comprises of 282 cycles of a realistic industrial assembly task. The annotations consist of 6 mid-level actions and an additional Null class. Five different test subjects performed the task without specific instructions on how to assemble the used car shielding. In this paper, we describe the dataset, set guidelines for using the data in supervised learning approaches, and analyze the labeling error caused by the labeler onto the dataset. We also compare different state-of-the-art neural networks to set the first benchmark and achieve a weighted F1score of 0.717.

Deploying navigation algorithms on an edge or cloud server according to the Software-as-a-Service paradigm has many advantages for autonomous mobile robots in indus-trial environments, e.g. cooperative planning and less onboard energy consumption. However, outsourcing corresponding real-time critical control functions requires a high level of reliability, which cannot be guaranteed either by modern wireless networks nor by the outsourced computing infrastructure. This work introduces redundancy concepts, which enable real-time capability within these uncertain infrastructures by providing redundant computation nodes, as well as robot-controlled switching between them. Redundancies can vary regarding their physical location, robot behavior during the switchover process and degree of activeness while quality of service concerning the primary controller is sufficient. In the case that fallback redun-dancies are not continuously active, when a disturbance occurs an initial state estimation of the robot pose has to be provided and an activation time has to be anticipated. To gain some insights on expected behavior, redundant computation nodes are deployed locally on the robot and on an outsourced computation node and consequently evaluated empirically. Quantitative and qualitative results in simulation and a real environment show that redun-dancies help to significantly improve the robot-trajectory within an unreliable network. Moreover, resource-saving redundancies, which are not continuously active, can robustly take over control by using an estimated state.

Energy efficiency is an emerging topic for companies in the industrial sector. The optimization of existing machines by advanced control is increasingly approached. For interconnected systems, we present a general procedure to include topology knowledge in an automated set-point optimization routine. This paper demonstrates an exemplary algorithm, applied to a parallel connection of two chillers for test purposes.

In this work we present our first prototype of a modular, low-cost end-of-arm concept for industrial robot applications. The goal is a faster, more flexible and cost-effective alternative compared to current industrial solutions. We use an embedded single-board computer and three cameras as a sensor base for the new system. The scope of robot application is supposed to as wide as possible (e.g. object detection, bin picking, assembly tasks and quality control tasks). We discuss some industry and low-cost-hardware solutions, introduce our system and deliver a proof of concept. Furthermore we use the system to accomplish a vision based pick and place task.

Neural networks have proven to be extraordinarily successful in many computer vision applications. But the approaches used to train neural networks require large datasets of annotated images, which requires a solid amount of human time to prepare those datasets. To facilitate the adoption of machine learning based technologies in industrial computer vision applications, this paper presents a two-step unsupervised learning approach for anomaly detection with further defect clusterization. In the first stage, the defects are not explicitly learned, but are interpreted as an anomaly or novelty based on the dataset of defect-free samples. In a second stage, the anomalies detected in the first stage are clustered in unsupervised manner and classified into meaningful categories by experts with process knowledge (e.g. critical or non-critical defect). This paper presents a first small dataset containing one industrial object with a complex shape. The object is made of aluminium and is shown both free of defects and defective. Based on this, recommendations are given for an acquisition setup for a large, extensive dataset. Most of the existing papers are studying the approaches for uniform surface (texture) inspection. The specifics of this research is to identify defects on rigid bodies, which exhibit highly non uniform texture in the image. State of the art methods were evaluated and improved to increase the classification accuracy. With a fine-tuned ResNet-18 it was possible to achieve 100% accuracy for defective and defect-free images.

Manufacturing processes provide large amounts of data which can be captured with current technology. Monitoring dashboards are visualization tools that help turn these data into knowledge in order to enhance production performance by supporting decision-making processes. However, configuring these dashboards can be a resource and time-consuming task when they must be tailored and customized among the various applications. The scope of this paper is to automate the configuration process of dashboards in order to ease their employment. In this fashion, end-users can experience a more efficient way of implementing monitoring dashboards on different cases. The idea of this work is to develop a dashboard system which provides autoconfiguration and adaptability. To achieve these goals, a modular automation concept is implemented. These features are demonstrated in a proof-of-concept called Automated Monitoring Dashboard (AMD) and its development is presented in this paper.

Optimizing the energy efficiency of machine tools automatically is promising. There are several metrics to be considered when it comes to automated optimization approaches in serial production which are especially quality, technical availability, and cycle time. These are not supposed to be impaired whereas they are indicated as a central obstacle. The measurements and the machine data show the actions happening in the machine which also leads to the data-driven traceability of machine states. This article presents a method to formulate the necessary expert knowledge to optimize the energy efficiency of a machine tool and is basically done by a decision tree which leads to a set of rules which will be explained in this article. This set of rules coordinate an optimization algorithm, which technically manipulates selected variables under the given rules. The development and is a result of a research which was done at the serial production of camshafts at the MB plant in Berlin.

For advanced control of technical systems reliable system identification methods are essential. The data-driven framework Sparse Identification of Nonlinear Dynamics (SINDy) by Kutz and Brunton is extended in order to tackle hysteresis-controlled systems. In order to gain robustness, a so called proximity hysteron is introduced. This paper presents this extension and documents experiments with simulations of an academic example and an industrial chiller system. A proof of concept is followed by experiments which show that strong nonlinearities as well as inadequate sampling rates can critically impair the algorithm.

Nowadays production industry and smart factory is dealing with methods of optimal resource load balancing and new types of flexible service-oriented strategies. It is seen crucial to adapt quickly to changes in manufacturing processes and new products or even integrate new hardware faster than the competition. Flexibility and Scalability can be improved by exchanging only a certain part of hardware and software without the need of touching all the other components. In this paper we present a methodical approach towards a typical use case in modern industrial robotic systems. The system consists of hardware components from different manufacturers which can be controlled and monitored separately by remote services. Those services can be combined to complex applications and integrate value added services. We show the independence and capability of exchangeable added value services running either centralized, decentralized, locally or remote. The experiments demonstrate how a process is improve by simply adding another service according to the Plug-and-Play paradigm. The service ensures the conditions of a computer vision system component to keep the reliability of the overall system workflow. In addition it will be demonstrated how system components could be virtualized in container-based cloud environments to save required on-board resources of the robotic system while keeping the whole system communication secure. Finally, results will be presented for different intercommunication scenarios.

Classic off- and online programming approaches offer different advantages but cannot provide intuitive programming as it would be needed for frequent reconfiguration of industrial robotic cells and their programs. While a simulation and therewith a collision control can be used offline, highest precision can be achieved by using online teach-in. We propose a system that combines the advantages of both offline and online programming. By creating a programming framework that is independent of in- and output devices we allow the use of augmented as well as virtual reality. Thus, enabling the user to choose for a programming technique that best suits his needs during different stages of programming. Bringing together both methods allows us to not only drastically reduce programming time but simultaneously increase intuitiveness of human robot interaction.

Part identification is relevant in many industrial applications, either for direct recognition of components or assemblies, either as a fully automated process or as an assistance system. Convolutional Neural Networks (CNNs) have proven their worth in image processing, especially in classification tasks. It therefore makes sense to use them for industrial applications. There are major problems with parts that look very similar and can only be identified by their size. In this paper we have considered a subset of screws that all conform to the same norm but are of different sizes. The implicit learning of the screw size is only possible if the images are taken in a fixed distance setup and larger screws are shown larger on the images. In this paper we show that CNNs are able to implicitly measure target objects with the help of reference objects and thus to integrate the object size into the learning process.

Deep Learning-based approaches for 3d object detection and 6d pose estimation typically require large amounts of labeled training data. Labeling image data is expensive and particularly the 6d pose information is difficult to obtain, as it requires a complex setup during image acquisition. Training with synthetic data is therefore very attractive. Large amounts of synthetic, labeled data can be generated, but it is not yet fully understood how certain aspects of data generation affect the detection and pose estimation performance. Our work therefore focuses on creating synthetic training data and investigating the effects on detection performance. We present two methods for data generation: rendering object views and pasting them on random background images, and simulating realistic scenes. The former is computationally simpler and achieved better results, but the detection performance is still very sensitive to small changes, e.g. the type of background images.

Industrial facilities are complex and heterogeneous systems in permanent technological change. The ambitions towards smart factories heighten the requirements for the flexible interconnection of various devices. These industrial entities are controlled, observed and optimized by many services. The tuning of process parameters of several linked components in order to boost the overall energy efficiency is one example of such services. AutomationML (AML) provides a hierarchical description language for industrial systems considering both structure and properties. An extension of the established standard allows for intuitive modeling of energy optimization problems. An approved energy performance indicator (EnPI) methodology was integrated in the libraries of AML in order to simplify and shorten the modeling procedure for the optimization task. The procedure is demonstrated using the example of an industrial cooling system.

The classical design of industrial robot controllers relies strongly on constant and stable cycle times for any closed loop operation. Meanwhile, decentralizing robot control into independent services and rolling them out to virtual machines and cloud instances is an upcoming trend. The advantage of the distributed robot control lies in achieving full flexibility of location independent service execution. In contrast, the internet connectivity leads to unknown latency conditions and resulting unpredictable cycle times. That is not well handled by standard feedback controllers like PID or P-PI cascades. In addition to the time-critical aspect, the performance of a closed -loop control relies on the accuracy of the underlying model. Modeling is often labor-intensive and time-consuming. This paper presents four different implementations of a non-model based modified Switching Active Disturbance Rejection Control (ADRC) in a robotic use case. The comparison of these four concepts reveals the strengths and weaknesses of the ADRC approach regarding controller performance under the assumption of full dynamics compensation with tough communication conditions. By extending the Switching-ADRC approach by a gain-scheduler to smooth the switching between the linear and non-linear ADRC description and a static adjustment of control-parameters related to the measured round-trip time, the presented control-concept is able to stabilize the high-dynamic, partly unstable robot-system without model-information. The stability of the modified Switching-ADRC is demonstrated empirically by extensive experiments with a distributed industrial robot joint position control.

In this paper we contribute to the research on facilitating industrial robot programming by presenting a concept for intuitive drag and drop like programming of pick and place tasks with Augmented Reality (AR). We propose a service-oriented architecture to achieve easy exchangeability of components and scalability with respect to AR devices and robot workplaces. Our implementation uses a HoloLens and a UR5 robot, which are integrated into a framework of RESTful web services. The user can drag recognized objects and drop them at a desired position to initiate a pick and place task. Although the positioning accuracy is unsatisfactory yet, our implemented prototype achieves most of the desired advantages to proof the concept.

This paper proposes a concept for human-robot interaction using techniques of virtual and augmented reality on mobile devices like cell phones and tablets or mixed reality devices like the HoloLens. By combining data received from real robots together with the perception and abilities of a human operator innovative applications are imaginable. Visualizing not only the current robot state but also the robots environment captured with different sensors and processed with both machine and human vision can lead to a rising percentage of robot assisted workplaces or robot installations. Since the visualization of and the interaction with the robotic application is not locally restricted new or improved use cases like remote maintenance or faster startup of industrial robots can be realized. Therefore, an architecture split into three functional units and an overview of our implementation is presented.

This paper is a work in progress report on a novel system for intuitive gesture based robot programming. The major contribution is the addition of an expert system into the robot programming process. The expert system uses static knowledge, such as seam types, and dynamic knowledge to reason about the intended process. The dynamic knowledge holds information about the environment and is derived from sensor data. The paper outlines the required components and the current state of development of a prototype system. An example of the inference process is given for a robot laser welding application.

We present a hybrid programming method for industrial robots combining advantages of manual haptic guidance of the end-effector and programming approaches using non-haptic pointing gestures for the spatial definition of poses and trajectories. Whereas the bare-hand spatial interaction can be implemented and performed cost- and time-efficiently but lacks accuracy, haptic-interaction is more time-consuming but it is used in a reduced manner in order to enable a highly-accurate refinement of target working poses. Additionally, the user is supported by a mobile Augmented Reality simulation providing spatial validation of the robot program, program management and transmission towards the robot controller. The implementation is realized by a compliance control based on a sensor mounted between flange and end-effector combined with our former introduced approach for spatial programming. We conducted a user study comparing Teach-In and Offline programming. The analysis shows a significant reduction of programming duration as well as a reduction of programming errors compared with Teach-In. Most participants favor the hybrid programming system. No significant differences for the programming duration could be determined between experts and non-experts. In comparison between haptic and non-haptic interaction, non-experts favor non-haptic interaction due to the higher intuitiveness of pointing gestures compared to direct physical interaction.

Information builds the base of modern logistics. In the past companies have started collecting basic parameters, but for the purpose of extended product-identification the existing approaches are not sufficient. In this paper we present a system for enhanced information acquisition based on inherent features of products. Our prototype includes a hardware installation as well as a software framework to acquire reliable parameters and images of items in a parallel process for the use of logistic applications. The consequent use of image processing obtains superior grade information. This was used to develop valuable applications, such as a reverse matching algorithm for products in logistic applications. Based on this highly potential system we present an outlook of further value-added services.

The potential of cloud manufacturing, i.e. using cloud- and service-oriented IT structures in industrial applications, goes far beyond the current state of technology. The modularization and dissolution of the “automation pyramid” leads from a hierarchical information and communication structure towards a highly cross-linked structure. The classic separation of the field level to cell and management level up to the factory level will become obsolete. The limiting factor for an extensive implementation of cloud platforms in production environments is located in missing standard interfaces between machinery, robots, production systems and the novel cloud infrastructures and services. Especially machinery without Ethernet-based communication interfaces cannot be integrated. To fill this gap we introduce our concept and three different connection scenarios for a connector layer in the use-case industrial control-as-a-service for cloud manufacturing. The scenarios include the utilization of Programmable Logic Controllers and micro-controllers as connector technologies as well as direct machine to cloud connection. We evaluate each scenario with a realistic experimental setup and present the results of our experiments.

In forensics, virtual reconstruction of shredded documents is a well-known problem. Semi-automatic document reconstruction systems are usually used for virtual reconstruction. Here a content feature extraction, a content feature representation and a 1:1-matching-method for the use in such reconstruction systems are presented. The content representation is given in the form of so-called abstract structure objects (ASO), which are calculated based on foreground information distributions and on color categories. The presented 1:1-matching-method calculates local optima and places these optima in a global context in relation to cut-edge-pair. Experiments were performed on different real-world-datasets with different foreground characteristics. We show the good discrimination power of the presented method for the use in reconstruction systems regardless of the type of foreground information.

This paper describes our concept for control of robots and machine tools through a private factory cloud which is dynamically extended by additional public resources. After discussing the opportunities and drawbacks of the virtualization of robot controllers (RC) and software programmable logic controllers (soft-PLC), we present first experiences with the setup of private and public infrastructures. Finally, we introduce our experimental design for investigating the requirements of machine control algorithms regarding availability, dependability and realtime capability in local and public configurations.

We propose an assistance system which observes a process involving humans and automatically takes measures in situations where ergonomic guidelines are violated. The system uses a Microsoft Kinect Sensor to capture a depth map of the scene. Using this sensory information, the system extracts working pose and assesses ergonomics in the current situation. If an ergonomically critical situation occurs, the system flexibly adjusts the height of a workpiece during a process in order to enable the worker to adapt a more comfortable pose. First experiments show the potential of the method but also reveal the remaining technological challenge in capturing the pose. As future work, we plan to extend our existing pose capturing method for the 3D case. Moreover, we plan to conduct studies in order to analyse the acceptance of such a system in production environment.

We propose an approach for integrated object and path demonstration. The main idea is to choose the motion of the robot depending on the object it grabs. When programming, an object is fixed at the robot's gripper. Path programming is done using haptic guidance. Afterwards, if a robot is handed over an object it knows before, it reproduces the path assigned to the object. Path programming and object recognition can be done with a single force/torque sensor. In our opinion, this way of programming is especially suited for creating pick and place tasks in frequently changing production environments where (re-)programming has to be done frequently.

In this paper an algorithm for the efficient computation of a rejection threshold for Bayes classification is discussed. A theoretical and a practical evaluation of the performance regarding the accuracy of the numerical computation for uni- and multimodal high-dimensional probability distributions is given. Additionally some observations regarding the dimensionality and the number of samples are shared.

This paper gives an introduction to the process of an experimental project in pertaining to the planning and integrating of a human-robot interactive stone carving system. The process includes an analysis of manual stone carving by a professional sculptor, a risk assessment of the projected robotic assistant system resulting in the safety concepts, a design of a tool carrier with force sensors and the human-machine interface. A standard industrial robot was selected and enhanced with extensive periphery to build up the intelligent tool carrier. Besides scientific results regarding work space and contact force suitable for human-robot collaboration, this paper presents the sculptural aspects within the experiment project. The professional scluptor evaluated the results as beyond manual processing.

Nowadays information and automation technology infrastructures, as used in Critical Infrastructures, manufacturing plants or distributed sensor networks, have become very complex and dependant to automation. This does not only demand for higher bandwidths and bit rates in connecting network technologies, furthermore major security issues arise as seen over the last years. The vision of secure information and automation infrastructures cannot be achieved only by securing single parts, like a Programmable Logic Controller or Personal Computer, without looking at the interplay of components. The following paper shows an approach to enhance security in these infrastructures by detecting unknown anomalies thoughout all levels of automation, including the operator interface.

We have developed a spatial programming system for industrial robots based on gestures and Augmented Reality. In this respect, we aim for a markerless and robust finger gesture recognition on the handheld devices. The user should be enabled to interact with virtual objects representing the robot program in front of the device. The great challenge is to be fast and robust in terms of diffuse background and changing environmental conditions. In this contribution, we present an image-based finger gesture recognition approach which is tailored to efficient implementation on mobile devices, e.g. Smartphones and Tablet PCs. For this purpose, we introduce an adaptive, rule-based skin classifier. The tracking is done by a specific particle filter: the Condensation algorithm. Due to further implemented improvements, we reduce the number of particles in order to reduce the computational time of the algorithm. The gesture recognition is implemented in an Android app, enabling the user to translate and rotate virtual object within an Augmented Reality application in real-time.

The presented spatial programming system provides an assistance system for online programming of industrial robots. A handheld device and a motion tracking system establish the basis for a modular 3D programming approach corresponding to different phases of robot programming: definition, evaluation and adaption. Static and dynamic gestures enable the program definition of poses, trajectories and tasks. The spatial evaluation is done using an Augmented Reality application on a handheld device. Therefore, the programmer is able to move freely within the robot cell and define the program spatially through gestures. The camera image of the handheld is simultaneously enhanced by virtual objects representing the robot program. Based on 3D motion tracking of human movements and a mobile Augmented Reality application, we introduce a novel kind of interaction for the adaption of robot programs. The programmer is enabled to interact with virtual program components through bare-hand gestures. Such sample forms of interaction include translation and rotation applicable to poses, trajectories or tasks representations. Finally, the program is adapted according to the gestural changes and can be transferred from the handheld device directly to the robot controler.

Energy conservation is a key aspect towards sustain-ability and is pursued by both, research and industry. To this end, we present measurements of the power consumption of two industrial robots. Based on a selected motion pattern the velocity and acceleration of the robot is varied and we show that the power is measurable with good repeatability. The energy of these motions is calculated from the power time series. Contrary to expectations it turns out that slow motions are not necessarily the most energy efficient. Finally, we show that the best strategy for energy conservation depends heavily on the robot model.

Gesture-based control systems for industrial robotics benefit from a natural form of human machine interaction. Among the high costs of such systems, there are high requirements for safety, robustness of the gesture recognition and general applicability in the industrial environment. Novel consumer electronic sensors provide a robust recognition, reliability and adequate accuracies at low costs. We present our work on gesture-based control and programming with an emphasis on a natural definition of complex trajectories. Besides standard online programming methods, we introduce a programming technique, which enables the worker to define a movement by his body parts, e.g. the hand. Imitating the human movement, the robot can move its end-effector relatively from a starting point.

In robot assisted rehabilitation the development of sophisticated training strategies using force and impedance control methods is an important field of research. The objective of the presented work is to provide a haptic control framework for end-effector based gait rehabilitation robots, which allows for the training of arbitrary foot trajectories of daily life and the design of sophisticated, patient-adaptive training environments. Due to the end-effector design principle the machine carries the patient's full bodyweight during the whole training cycle which implies excessively high forces at the interaction ports. To allow for a guided movement under the described conditions the machine features powerful drives and a massive, stiff mechanical structure. This results in exceptionally high device inertia comparable to heavy-duty industrial robots and normally not common in haptic displays. To ensure stable interaction with a passive user and a known range of stiff and viscose virtual environments an adaptive virtual coupling controller is developed. To minimize the complexity of design procedure, a linear, discrete-time coupled and contact transition stability analysis is derived, taking into account the known target range of environments. The developed haptic controller was implemented and tested experimentally on the robotic walking simulator HapticWalker. It was shown to interact stably with virtual environments within the defined stiffness bounds. Further a set of lower target impedance parameters was evaluated in free motion without contact. An algorithm for the adaptation of the target damping was suggested to improve transparency during free motion while still ensuring a stable contact transition and interaction with stiff environment objects. The feasibility of the adaptive coupling controller was proven in free motion and walking experiments on the HapticWalker with the simulation static and dynamic virtual environments.

Rising demands to smaller lot sizes and more flexibility in production processes inspired the idea of self-organising production by the use of “intelligent” parts. This is achieved by self-sustaining wireless nodes - Process-eGrains - to transfer the production control and logic from a centralised approach to a distributed logic. This paper will show the requirements in an industrial environment from an application perspective to these nodes. We present the middleware design implemented on an agent-driven architecture. Abstracting from different hardware it provides a unified interface for storing and processing information independently and locally and negotiate among each other. The application is exemplified through a part driven assembly.

In recent years a swarm-based optimization methodology called particle swarm optimization (PSO) has developed. PSO is highly explorative and primarily used in function optimization. This paper proposes a swarm-based learning algorithm based on PSO which is able to discover trees in tree spaces. Particles are flying through a tree space forming flocks around peaks of a fitness function. Because it inherits the explorative property of PSO, it needs only few evaluations to find suitable trees.

This paper presents a new haptic locomotion interface, which comprises two programmable foot platforms with permanent foot machine contact. It is designed as a scalable and modular system with unit-by-unit extensibility offering up to six plus one degrees of freedom (DOF) per foot. The basic setup comprises three DOF per foot in the sagittal plane.The machine is based on a rigid hybrid parallel-serial robot kinematics structure. It is equipped with electrical direct drive motors, enabling highly dynamic footplate motions. For contact force measurement, six DOF force/torque sensors are mounted under each foot platform. The system was developed for major application in gait rehabilitation, hence great importance was attached to the incorporation of maximum passive and active security measures for machine users and medical operating personnel.The simulator is able to perform walking trajectories with speeds of up to 5 km/h and 120 steps/min. The system is able to simulate not only slow and “smooth” trajectories like walking on an even floor, up/down staircases, but also foot motions like walking on rough ground or even stumbling or sliding, which require high system dynamics.The machine is controlled by a self-developed full-featured robot control whose soft and hardware is based on up-to-date industrial standards and interfaces. The robot control software is based on RTLinux and runs on an industrial PC. The real-time motion generator includes a newly developed Fourier-based algorithm for the interpolation of natural cyclic walking trajectories. For the implementation of asynchronous events (e.g., sliding, stumbling), the controller comprises especially developed algorithms for automatic motion override adaptation. Different modes of haptic behavior needed for gait rehabilitation, ranging from full foot support during swing phase to completely passive behavior, are currently under development.Intuitive and safe machine operation by nontechnical personnel such as clinicians and physiotherapists is achieved via a separate Windows-based graphical user interface software comprising different window areas for machine programming and operation, real-time off-line simulation and online data visualization in two and three dimensions has been developed as well.A working prototype of the system has been built and tested successfully, including all soft and hardware components. Although the machine has been designed and built for major application in gait rehabilitation, its range of applicability is not limited to this area. It could be integrated into any setup requiring a highly dynamic haptic foot interface and permanent foot machine contact if needed.