In Fourier-domain optical coherence tomography (FD-OCT) imaging, saturation of the data acquisition chain leads to loss of information by clipping the spectral interferograms. When reconstructed, affected depth profiles are degraded by bright axial lines or periodical patterns which mask the underlying information and thereby affect clinical utility. In this work, we propose an approach to faithfully recover the distorted information from saturated FD-OCT scans based on physical insights. For this purpose, genesis and appearance of saturation artifacts are analyzed, revealing a wavelength-dependency within individual interferograms. We propose to use this knowledge in a dual manner: Firstly, we present a simulation model that can dynamically generate realistically saturated interferograms from clean counterparts. This physics-based model allows training neural networks for artifact removal solely on synthesized image pairs. Secondly, to make the wavelength-dependency of artifacts directly accessible, we propose a multi-input, single-output (MISO) network framework. In addition to full B-scans, MISO receives images reconstructed from various spectral sub-windows. Our experiments confirm successful generalization of networks trained on simulated training data to real-world artifact removal in ophthalmic anterior segment imaging. This includes reflexes of various origin in diagnostic or intra-surgical imaging, such as different tissue structures or surgical instruments. Furthermore, a comparison of the proposed multi-input network to single-input baselines reveals consistent performance and reliability gains on both swept-source and spectral domain datasets.

Optical Coherence Tomography Angiography (OCTA), a functional extension of OCT, has the potential to replace most invasive fluorescein angiography (FA) exams in ophthalmology. So far, OCTA's field of view is however still lacking behind fluorescence fundus photography techniques. This is problematic, because many retinal diseases manifest at an early stage by changes of the peripheral retinal capillary network. It is therefore desirable to expand OCTA's field of view to match that of ultra-widefield fundus cameras. We present a custom developed clinical high-speed swept-source OCT (SS-OCT) system operating at an acquisition rate 8-16 times faster than today's state-of-the-art commercially available OCTA devices. Its speed allows us to capture ultra-wide fields of view of up to 90 degrees with an unprecedented sampling density and hence extraordinary resolution by merging two single shot scans with 60 degrees in diameter. To further enhance the visual appearance of the angiograms, we developed for the first time a three-dimensional deep learning based algorithm for denoising volumetric OCTA data sets. We showcase its imaging performance and clinical usability by presenting images of patients suffering from diabetic retinopathy.

Many real-world big data applications in domains like finance, telecommunication and manufacturing rely on the detection of exceedingly rare patterns in large time series data sets. In principle, machine learning models can be trained to detect and classify such patterns. However, these models often lack the necessary robustness for practical applications and do not generalize well in production. Additionally, their intransparent decision-making hampers systematic debugging and improvement. Time series shapelets are a popular data mining primitive that can be used to extract a shape-based feature representation from the data. However, existing algorithms do not adequately consider the robustness and redundancy of these features. While these drawbacks can be compensated if sufficient labeled data is available, this is not possible for highly imbalanced data. We propose alterations to the current state-of-the-art shapelet algorithm that consider the margin of separation and the multivariate dependencies between the extracted features. This results in more robust and diverse features, which in turn translates to higher classification accuracy. We compare our algorithm to the current state-of-the-art using a public benchmark data set. Additionally, we showcase its applicability to highly imbalanced data using a suitable data set from the manufacturing domain.

Virtual reality is a powerful tool for industrial applications. The article at hand addresses designers of industrial virtual environments. It summarizes key aspects to design immersive and coherent virtual environments. Furthermore, relevant influencing factors for a high quality virtual environment and tools to quantify this quality are presented. So far, a methodology to design, evaluate, and transfer knowledge from virtual environments into reality has been missing and is of high value for industrial applications. The proposed methodical approach includes the steps application analysis, technology selection and integration, design of virtual environment, evaluation of simulator quality, as well as discussion of the real-world validity. The method is shown on the example of a virtual human-robot working cell used to analyze the human perception of robot behavior during mutual assembly processes. The quality of the virtual environment is evaluated to be adequate for those purposes and the transfer of knowledge gained in virtuality on a corresponding real-world application is discussed. To the best of our knowing a system like the presented one, including full-body tracking, finger tracking, a virtual avatar and a head-mounted display has not been used for industrial use cases and human-robot cooperation before.

The ability to synchronize expectations among human-robot teams and understand discrepancies between expectations and reality is essential for human-robot collaboration scenarios. To ensure this, human activities and intentions must be interpreted quickly and reliably by the robot using various modalities. In this paper we propose a multimodal recognition system designed to detect physical interactions as well as nonverbal gestures. Existing approaches feature high post-transfer recognition rates which, however, can only be achieved based on well-prepared and large datasets. Unfortunately, the acquisition and preparation of domain-specific samples especially in industrial context is time consuming and expensive. To reduce this effort we introduce a weakly-supervised classification approach. Therefore, we learn a latent representation of the human activities with a variational autoencoder network. Additional modalities and unlabeled samples are incorporated by a scalable product-of-expert sampling approach. The applicability in industrial context is evaluated by two domain-specific collaborative robot datasets. Our results demonstrate, that we can keep the number of labeled samples constant while increasing the network performance by providing additional unprocessed information.

Diagnosis and treatment guidance are aided by detecting relevant biomarkers in medical images. Although supervised deep learning can perform accurate segmentation of pathological areas, it is limited by requiring a priori definitions of these regions, large-scale annotations, and a representative patient cohort in the training set. In contrast, anomaly detection is not limited to specific definitions of pathologies and allows for training on healthy samples without annotation. Anomalous regions can then serve as candidates for biomarker discovery. Knowledge about normal anatomical structure brings implicit information for detecting anomalies. We propose to take advantage of this property using Bayesian deep learning, based on the assumption that epistemic uncertainties will correlate with anatomical deviations from a normal training set. A Bayesian U-Net is trained on a well-defined healthy environment using weak labels of healthy anatomy produced by existing methods. At test time, we capture epistemic uncertainty estimates of our model using Monte Carlo dropout. A novel post-processing technique is then applied to exploit these estimates and transfer their layered appearance to smooth blob-shaped segmentations of the anomalies. We experimentally validated this approach in retinal optical coherence tomography (OCT) images, using weak labels of retinal layers. Our method achieved a Dice index of 0.789 in an independent anomaly test set of age-related macular degeneration (AMD) cases. The resulting segmentations allowed very high accuracy for separating healthy and diseased cases with late wet AMD, dry geographic atrophy (GA), diabetic macular edema (DME) and retinal vein occlusion (RVO). Finally, we qualitatively observed that our approach can also detect other deviations in normal scans such as cut edge artifacts.

Collaboration between robots and humans requires communicative skills on both sides. The robot has to understand the conscious and unconscious activities of human workers. Many state-of-the-art activity recognition algorithms with high performance rates on existing benchmark datasets are available for this task. This paper re-evaluates appropriate architectures in light of human work activity recognition for working cells in industrial production contexts. The specific constraints of such a domain is elaborated and used as prior knowledge. We utilize state-of-the-art algorithms as spatiotemporal feature encoders and search for appropriate classification and fusion strategies. Furthermore, we combine keypoint-based with appearance-based approaches to a multi-stream recognition system. Due to data protection rules and the high effort of data annotation within industrial domains only small datasets are available that reflect production aspects. Therefore, we use transfer learning approaches to reduce the dependency on data volume and variance in the target domain. The resulting recognition system achieves high performance for both singular person action and human-object interaction.

The identification and quantification of markers in medical images is critical for diagnosis, prognosis, and disease management. Supervised machine learning enables the detection and exploitation of findings that are known a priori after annotation of training examples by experts. However, supervision does not scale well, due to the amount of necessary training examples, and the limitation of the marker vocabulary to known entities. In this proof-of-concept study, we propose unsupervised identification of anomalies as candidates for markers in retinal optical coherence tomography (OCT) imaging data without a constraint to a priori definitions. We identify and categorize marker candidates occurring frequently in the data and demonstrate that these markers show a predictive value in the task of detecting disease. A careful qualitative analysis of the identified data driven markers reveals how their quantifiable occurrence aligns with our current understanding of disease course, in early- and late age-related macular degeneration (AMD) patients. A multi-scale deep denoising autoencoder is trained on healthy images, and a one-class support vector machine identifies anomalies in new data. Clustering in the anomalies identifies stable categories. Using these markers to classify healthy-, early AMD- and late AMD cases yields an accuracy of 81.40%. In a second binary classification experiment on a publicly available data set (healthy versus intermediate AMD), the model achieves an area under the ROC curve of 0.944.

Acute Myeloid Leukaemia (AML) is a rare type of childhood acute leukaemia. During treatment, the assessment of the number of cancer cells is particularly important to determine treatment response and consequently adapt the treatment scheme if necessary. Minimal Residual Disease (MRD) is a diagnostic measure based on Flow CytoMetry (FCM) data that captures the amount of blasts in a blood sample and is a clinical tool for planning patients' individual therapy, which requires reliable blast identification. In this work we propose a novel semi-supervised learning approach, which is acquired whenever large amounts of unlabeled data and only a small amount of annotated data is available. The proposed semi-supervised learning approach is based on Wasserstein Generative Adversarial Network (WGAN) latent space embeddings learned in an unsupervised fashion and a simple Fully connected Neural Network (FNN) trained on labeled data leveraging the learned embedding. We apply our proposed learning approach for semi-supervised classification of blasts vs. non-blasts. We compare our approach with two baseline approaches, 1) semi-supervised learning based on Principal Component Analysis (PCA) embedding, and 2) a deep FNN that is trained only on the annotated data without leveraging an embedding. Results suggest that our proposed semi-supervised WGAN embedding outperforms semi-supervised learning based on PCA embeddings and if only small amounts of annotated data is available it even outperforms an FNN classifier.

In smart manufacturing environments robots collaborate with human operators as peers. They even share the same working space and time. An intuitive interaction with different input modalities is decisive to reduce workload and training periods for collaboration. We introduce our interaction system that is able to recognize gestures, actions and objects in a typical smart working scenario. As key aspect, this article considers an empirical investigation of input modalities (touch, gesture), individual differences (performance, recognition rate, previous knowledge) and boundary conditions (level of automation) on user experience. Therefore, answers from 31 participants within two experiments are collected. We show that the arrangement of the human-robot collaboration (input modalities, boundary conditions) has a significant effect on user experience in real-world environments. This effect and the individual differences between participants can be measured utilizing recognition rates and standardized usability questionnaires.

This article discusses the relevance of the motion behavior and adaptation of a collaborative robot for human-robot cooperation. Two experiments on cooperative assembly are shown. First, a human-human experiment with defined test conditions evaluates the aspects of distance, nearest body part, and predictability as significant. Second, a human-robot experiment shows that fixed trajectories and conservative dynamic parameters lead to a quick gain of confidence of the participants. Besides, the data shows that a realistic use case with complex tasks is key to evaluate the impact of motion parameters.

Seamless human-robot collaboration depends on high non-verbal behaviour recognition rates. To realize that in real-world manufacturing scenarios with an ecological valid setup, a lot of effort has to be invested. In this paper, we evaluate the impact of spontaneous inputs on the robustness of human-robot collaboration during gesture-based interaction. A high share of these spontaneous inputs lead to a reduced capability to predict behaviour and subsequently to a loss of robustness. We observe body and hand behaviour during interactive manufacturing of a collaborative task within two experiments. First, we analyse the occurrence frequency, reason and manner of human inputs in specific situations during a human-human experiment. We show the high impact of spontaneous inputs, especially in situations that differ from the typical working procedure. Second, we concentrate on implicit inputs during a real-world Wizard of Oz experiment using our human-robot working cell. We show that hand positions can be used to anticipate user needs in a semi-structured environment by applying knowledge about the semi-structured human behaviour which is distributed over working space and time in a typical manner.

Within this paper a modeling, identification and control technique for an antagonistically actuated joint consisting of two pneumatically actuated muscles is presented. The antagonistically actuated joint acts as a test bench for control architectures which are going to be used to control an exoskeleton within a telerobotic system. A static and dynamic model of the muscle and the joint is derived and the parameters of the models are identified using a least-squares algorithm. The control architecture, consisting of a inner pressure and an outer position controller is presented. The pressure controller is evaluated using switching valves compared against proportional valves.

In this paper, a new method for a minimum-error pose estimation of cylinder axes based on apparent contour line features from multiple views is presented. Novel model equations for both single and particularly multiple views are derived, and based upon these, an iterative algorithm for least squares fitting the model to imaged cylinder contour line features is introduced. The good performance and fast convergence of the proposed algorithm is shown by solving exemplary fitting problems.

Manual tasks in industrial production are often monotonous, leading to a decrease in concentration and motivation of the worker and thus to deficiencies in the products. With quality as well as performance requirements getting more and more stringent, workers need additional support by their work environment. We developed a novel approach providing worker assistance and inline quality assurance for manual workplaces. The prototypical system Smart Workbench (SWoB) uses a multimodal sensor interface consisting of a 3D depth sensor in combination with a 2D camera to control quality aspects of the product and track the work progress. The bidirectional flow of information is handled via an image processing driven gestural interface and the displaying of advice directly on the work surface. In this paper the developed system is introduced and the current state of evaluating its industrial usage with a manual quality control and packaging task reported.

All mammals but humans use whiskers in order to rapidly acquire information about objects in the vicinity of the head. Collisions of the head and objects can be avoided as the contact point is moved from the body surface to the whiskers. Such a behavior is also highly desirable during many robot tasks such as for human-robot interaction. Using novel capacitive proximity sensors, robots sense when they approach a human (or an object) and react before they actually collide with it. We propose a sensor and control concept that mimics the behavior of whiskers by means of capacitive sensors. Major advantages are the absence of physical whiskers, the absence of blind spots and a very short response time. The sensors are flexible and thin so that they feature skin-like properties and can be attached to various robotic link and joint shapes. In comparison to capacitive proximity sensors, the proposed virtual whiskers offer better sensitivity towards small conductive as well as non conductive objects. Equipped with the new proximity sensors, a seven-joint robot for humanrobot interaction tasks shows the efficiency and responsiveness of our concept.

The objectives of this work are the design of an experimental methodology for evaluation of the behavior of human or animal joints under physiological load conditions and the setup of robotic systems using force-torque control (FTC) to apply these realistic force-motion trajectories to cadaveric joints. In this context also the benefits of FTC for biomechanical research were aimed to show. Therefore, integrated methods using FTC have been developed for application of test series to diarthrodial joints. In the work reported here, these methods were successfully deployed to dissected porcine knee joints.

This paper presents an approach to fast object manipulation by dextrous re-grasping of multi-fingered hands. The approach is based on a real-time grasping force optimization (GFO) algorithm and a fingertip impedance control scheme. Both the controller and the GFO make use of 6D contact force sensor data at run-time. The latter keeps the contact forces as small as possible while considering friction limits at the contact points during (re)grasping. The impedance controller is used to impose the optimized contact forces onto the grasped object while simultaneously enabling active control of the fingertip positions. Experiments demonstrate the robustness of the approach and the increase in task speed during multi-fingered manipulation.

This paper presents an approach towards modeling and accurate dynamical simulation of multi-fingered dextrous manipulation. A hybrid discrete-continuous systems approach allows one to integrate time-driven mechanical characteristics and discrete-event aspects due to varying contact situations into one comprehensive modeling framework. A formal description is derived which can straightforwardly be embedded into a simulation environment like e.g. MATLAB. Dynamical simulations of a four-fingered hand manipulating an object demonstrate the high accuracy of our approach.

In this paper we present a control approach to dextrous manipulation based on a hybrid (discrete-continuous) dynamical system model. The discrete event aspect is the grasp state specifying whether fingers are in contact or not. The three components of the proposed discrete-continuous control approach to (re)grasping are a hybrid planning scheme for regrasping and discrete grasp state error compensation, an efficient grasping force optimization algorithm, and a variable structure grasp impedance controller. Experimental results are presented to validate the approach.

We present an algorithm which allows qualitative and quantitative analysis of the general force closure property of a grasp when fingertip contact forces are limited by unilateral friction constraints. It is applied to grasp synthesis during regrasping sequences. Contact points for moving fingers which result in structurally stable grasps are calculated. The proposed method is suitable for two and three dimensional grasps with an arbitrary number of fingers. The approach is validated by numerical examples and in dynamical simulations.

We present an approach to compensate for discrete contact state errors in regrasping tasks with multi-fingered robotic hands. An adaptation of reference trajectories connected with a hybrid discrete continuous control architecture allows stable grasps even if moving fingers cannot contact at the desired position on a grasped object. We propose a robust control setup that allows multiple regrasping tasks of objects to be performed using this compensation strategy and contact force optimization. Experimental results with the hydraulic robotic hand of the TUM verify the proposed approach and illustrate the increase in robustness for multi-fingered manipulation.

We propose an approach to model a robotic hand as a hybrid discrete-continuous dynamical system. The discrete event aspect is the grasp state specifying whether fingers are in contact or not. The hybrid approach yields a comprehensive model of a closed-hoop controller for dextrous (re)grasping. Insights in the closed-loop behavior and compensation strategies for discrete state errors due to object geometry modeling errors are presented. Simulations validate the hybrid systems approach to regrasping.

Buss et al. previously (1995, 1996) presented a real-time applicable grasping force optimization scheme for stable dextrous grasping. The authors present a novel approach to planning multifingered regrasping based on a modification of the optimization algorithm. This novel algorithm allows for smooth grasping force transitions during regrasping tasks. The optimization scheme is implemented combined with an impedance control law. Simulation results show the efficiency and simplicity of our approach to regrasping.