The emergence of Digital Twins (DT) in Industry 4.0 has enabled the decision support systems taking advantage of more effective recommendation systems (RS). Despite the RS's growing popularity and ability to support decision-makers, these face two significant challenges, cold-start and data sparsity, which limits the system's capability to provide effective and accurate decision support. This paper aims to address these issues by conducting a literature review, analysing the current research landscape, and identifying the main enabling methods, algorithms, and similarity measures to mitigate these challenges. The performed analysis enables the point out of future research directions for developing effective and accurate RS that empower decision-makers.

Visual perception enables robots to perceive the environment. Visual data is processed using computer vision algorithms that are usually time-expensive and require powerful devices to process the visual data in real-time, which is unfeasible for open-field robots with limited energy. This work benchmarks the performance of different heterogeneous platforms for object detection in real-time. This research benchmarks three architectures: embedded GPU—Graphical Processing Units (such as NVIDIA Jetson Nano 2 GB and 4 GB, and NVIDIA Jetson TX2), TPU—Tensor Processing Unit (such as Coral Dev Board TPU), and DPU—Deep Learning Processor Unit (such as in AMD/Xilinx ZCU104 Development Board, and AMD/Xilinx Kria KV260 Starter Kit). The authors used the RetinaNet ResNet-50 fine-tuned using the natural VineSet dataset. After the trained model was converted and compiled for target-specific hardware formats to improve the execution efficiency. The platforms were assessed in terms of performance of the evaluation metrics and efficiency (time of inference). Graphical Processing Units (GPUs) were the slowest devices, running at 3 FPS to 5 FPS, and Field Programmable Gate Arrays (FPGAs) were the fastest devices, running at 14 FPS to 25 FPS. The efficiency of the Tensor Processing Unit (TPU) is irrelevant and similar to NVIDIA Jetson TX2. TPU and GPU are the most power-efficient, consuming about 5 W. The performance differences, in the evaluation metrics, across devices are irrelevant and have an F1 of about 70 % and mean Average Precision (mAP) of about 60 %.

The research about the digital twin concept is growing worldwide, especially in the industrial sector, due to the increasing digitisation level associated to Industry 4.0. The application of the digital twin concept improves performance of a system by implementing monitoring, diagnosis, optimisation, and decision support actions. In particular, the decision-making process is very time consuming since the decision-maker is presented with hundreds of different scenarios that can be simulated and assessed in a what-if perspective. Bearing this in mind, this paper proposes to integrate a digital twin-based what-if simulation with a recommendation system to improve the decision-making cycle. The recommendation system is based on a reinforcement learning technique and takes user knowledge of the system into consideration and trust in the system recommendation. The applicability of the proposed approach is presented in an assembly line case study for recommending the best configurations for the system operation, in terms of the optimal number of AGVs (Autonomous Guided Vehicles) in various scenarios. The achieved results show its successful application and highlight the benefits of using AI-based recommendation systems for what-if simulation in digital twin systems.

Robotics competitions are environments that foster teamwork, AI, and technology development by encouraging students, researchers, and academics to test their solutions against each other. These competitions often challenge the competitors’ prototypes with tasks specifically designed to benchmark them with the current optimal solutions. During the prototype stages of a robot, the development costs and time spent are often higher than other stages, as changes in the prototype are frequent. Simulation is often used to reduce these variables as it allows flexibility in all development stages before transitioning to the real scenario. However, a digital twin can be used to increase even further flexibility and effectiveness. Digital twins are virtual representations of real assets, providing replication and prediction of real scenario events, and real-time monitoring of the real object. Thus, this paper presents the development of a digital twin of an automatic guided vehicle (AGV) to the Robot@Factory Lite competition and the tests performed to validate the approach.

The digital transformation that is on-going worldwide, and triggered by the Industry 4.0 initiative, has brought to the surface new concepts and emergent technologies. One of these new concepts is the Digital Twin, which recently started gaining momentum, and is related to creating a virtual copy of the physical system, providing a connection between the real and virtual systems to collect and analyze and simulate data in the virtual model to improve the performance of the real system. The benefits of using the digital twin approach is attracting significant attention and interest from research and industry communities in the last few years, and its importance will increase in the upcoming years. Having this in mind, this paper surveys and discusses the digital twin concept in the context of the 4th industrial revolution, particularly focusing the concept and functionalities, the associated technologies, the industrial applications and the research challenges. The applicability of the digital concept is illustrated by the virtualisation of an UR3 collaborative robot which used the V-REP simulation environment and the Modbus communication protocol.

Abstract Integration in the world of sport is one way for individuals with disabilities or motor disorders to feel more socially integrated, independent, and confident. Boccia is a Paralympic sport, which is increasingly getting more attention around the world. These facts have contributed to the objectives of this work. Including it in the serious games category enables to develop and rehabilitate the cognitive capabilities. The main focus was BC3 classification athletes (users with limited motor characteristics that require the use of an assistive device—a ramp, in this case). This paper describes a realistic Boccia game simulator adapted for people with disabilities that integrates a set of features that includes real physics and social features. These features can be used to enhance the interest of nonpractitioners of the sport and to improve the training conditions. The official Boccia regulation was added to the design of the simulator. The usability and approximation to the reality of the simulator were tested and validated based on the tests performed and data collected via a survey of users with no motor or psychological disorders. Realism and usability rating was almost excellent, and good results were achieved at the assessment of the game experience.

In this paper we introduce the problem of planning for perception of a target position. Given a sensing target, the robot has to move to a goal position from where the target can be perceived. Our algorithm minimizes the overall path cost as a function of both motion and perception costs, given an initial robot position and a sensing target. We contribute a heuristic search method, PA*, that efficiently searches for an optimal path. We prove the proposed heuristic is admissible, and introduce a new goal state stopping condition.

We introduce in this paper visibility maps for robots of any shape, representing the reachability limit of the robot's motion and sensing in a 2D gridmap with obstacles. The brute-force approach to determine the optimal visibility map is computationally expensive, and prohibitive with dynamic obstacles. We contribute the Robot-Dependent Visibility Map (RDVM) as a close approximation to the optimal, and an effective algorithm to compute it. The RDVM is a function of the robot's shape, initial position, and sensor model. We first overview the computation of RDVM for the circular robot case in terms of the partial morphological closing operation and the optimal choice for the critical points position. We then present how the RDVM for any-shape robots is computed. In order to handle any robot shape, we introduce in the first step multiple layers that discretize the robot orientation. In the second step, our algorithm determines the frontiers of actuation, similarly to the case of the the circular robot case. We then derive the concept of critical points to the any-shape robot, as the points that maximize expected visibility inside unreachable regions. We compare our method with the ground-truth in a simulated map compiled to capture a variety of challenges of obstacle distribution and type, and discuss the accuracy of our approximation to the optimal visibility map.

This paper presents a new marker for robot programming by demonstration through motion imitation. The device is based on high intensity LEDs (light emission diodes) which are captured by a pair of industrial cameras. Using stereoscopy, the marker supplies 6-DoF (degrees of freedom) human wrist tracking with both position and orientation data. We propose a robust technique for camera and stereo calibration which maps camera coordinates directly into the desired robot frame, using a single LED. The calibration and tracking procedures are thoroughly described. The tests show that the marker presents a new robust, accurate and intuitive method for industrial robot programming. The system is able to perform in real-time and requires only a single pair of industrial cameras though more can be used for improved effectiveness and accuracy.

Intelligent wheelchairs can become an important solution to assist physically impaired individuals who find it difficult or impossible to drive regular powered wheelchairs. However, when designing the hardware architecture several projects compromise the user comfort and the wheelchair normal usability in order to solve robotic problems. In this paper we describe the main concepts regarding the design of the IntellWheels intelligent wheelchair. Our approach has a user-centered perspective, in which the needs and limitations of physically impaired users are given extensive attention at each stage of the design process. Finally, our design was evaluated through a public opinion assessment. A statistical analysis suggested that the design was effective to mitigate the visual and ergonomic impacts caused by the addition of sensorial and processing capabilities on the wheelchair.

The main objective of this project was to evaluate the sensing system for person detection in the scope of its integration in a mobile robotic platform for Ambient Assisted Living. Two sensors were considered, a camera and a depth sensor. For the camera (2D), 3 different algorithms were implemented: face detection, tracking and recognition. For the depth sensor the whole body detection was tested using skeleton tracking. The results showed that the face detection and recognition algorithms had a very small range, and the face tracking demonstrated to reach a higher distance. However, the latter exhibited poor results when confronted with illumination variations. The skeleton tracking algorithm provided good results, being capable of tracking relatively close to the sensor and up to 3 meters distance. Thus, the 2D face recognition can be used for short distances to identify the person, while the 3D skeleton tracking can be appropriated for distant tracking of the person. This work showed that the integration of these sensing systems, in a robotic platform, can make a robust robot capable of human interaction in home environments.

Maximizing the performance of cooperative perception of a tracked target by a team of mobile robots while maintaining the team's formation is the core problem addressed in this work. We propose a solution by integrating the controller and the estimator modules in a formation control loop. The controller module is a distributed non-linear model predictive controller and the estimator module is based on a particle filter for cooperative target tracking. A formal description of the integration followed by simulation and real robot results on two different teams of homogeneous robots are presented. The results highlight how our method successfully enables a team of homogeneous robots to minimize the total uncertainty of the tracked target's cooperative estimate while complying with the performance criteria such as keeping a pre-set distance between the team-mates and/or the target and obstacle avoidance.

New and more natural human-robot interfaces are of crucial interest to the evolution of robotics. This paper addresses continuous and real-time hand gesture spotting, i.e., gesture segmentation plus gesture recognition. Gesture patterns are recognized by using artificial neural networks (ANNs) specifically adapted to the process of controlling an industrial robot. Since in continuous gesture recognition the communicative gestures appear intermittently with the non-communicative, we are proposing a new architecture with two ANNs in series to recognize both kinds of gesture. A data glove is used as interface technology. Experimental results demonstrated that the proposed solution presents high recognition rates (over 99% for a library of ten gestures and over 96% for a library of thirty gestures), low training and learning time and a good capacity to generalize from particular situations.

This paper introduces a new approach to 3-D position estimation from acceleration data, i.e., a 3-D motion tracking system having a small size and low-cost magnetic and inertial measurement unit (MIMU) composed by both a digital compass and a gyroscope as interaction technology. A major challenge is to minimize the error caused by the process of double integration of accelerations due to motion (these ones have to be separated from the accelerations due to gravity). Owing to drift error, position estimation cannot be performed with adequate accuracy for periods longer than few seconds. For this reason, we propose a method to detect motion stops and only integrate accelerations in moments of effective hand motion during the demonstration process. The proposed system is validated and evaluated with experiments reporting a common daily life pick-and-place task.

Most of industrial robots are still programmed using the typical teaching process, through the use of the robot teach pendant. In this paper is proposed an accelerometer-based system to control an industrial robot using two low-cost and small 3-axis wireless accelerometers. These accelerometers are attached to the human arms, capturing its behavior (gestures and postures). An Artificial Neural Network (ANN) trained with a back-propagation algorithm was used to recognize arm gestures and postures, which then will be used as input in the control of the robot. The aim is that the robot starts the movement almost at the same time as the user starts to perform a gesture or posture (low response time). The results show that the system allows the control of an industrial robot in an intuitive way. However, the achieved recognition rate of gestures and postures (92%) should be improved in future, keeping the compromise with the system response time (160 milliseconds). Finally, the results of some tests performed with an industrial robot are presented and discussed.

This paper describes a joint trajectory optimized controller for a humanoid robot simulator following the real robot characteristics. As simulation is a powerful tool for speeding up the control software development, the proposed accurate simulator allows to fulfil this goal. The simulator, based on the Open Dynamics Engine and GLScene graphics library, provides instant visual feedback. The proposed simulator, with realistic dynamics, allows to design and test behaviours and control strategies without access to the real hardware in order to carry out research on robot control without damaging the real robot in the early stages of the development. The joints controller techniques, such as acceleration, speed and energy consumption minimization are discussed and experimental results are presented in order to validate the proposed simulator.

Localization is essential to modern autonomous robots in order to enable effective completion of complex tasks over possibly large distances in low structured environments. In this paper, a extended Kalman filter is used in order to implement self-localization. This is done by merging odometry and localization information, when available. The used landmarks are colored poles that can be recognized while the robot moves around performing normal tasks. This paper models measurements with very different characteristics in distance and angle to markers and shows results of the self-localization method. Results of simulations and real robot tests are shown.