Today, intelligent robotic manufacturing systems are reshaping the production industry. Using robots as actuators, multi-source sensors for perception, and Artificial Intelligence (AI) as decision-making systems, they can perform routine manufacturing tasks, surpassing the capabilities of traditional hard-programmed Computer Numerical Control (CNC) machinery. One specific challenge in footwear manufacturing is sole deburring, traditionally done manually by skilled workers. This paper focuses on developing a robust path-planning pipeline, comprising vision-based and Learning from Demonstrations (LfD) modules for autonomous deburring of soles. The vision-based module exploits Deep Learning (DL) techniques to handle key challenges such as precise segmentation of different soles types across diverse scenarios despite potential occlusions. Additionally, a novel method for burrs identification has been developed leveraging image processing and optimization techniques. Determining the optimal cutting tool orientation during sole deburring relies on human experience. The LfD module aims to impart this knowledge to the robot from videos of expert demonstrations, requiring adaptability to every new incoming sole that needs deburring. Experimental results showcase the method’s performance and flexibility, underlining the potential to advance the field of the proposed approach.

Labour shortage, difficulties in labour management, the digitalization of fruit production pipeline to reduce the fruit production costs have made robotic systems for selective harvesting of strawberries an important industry and academic research. One of the important components of such technologies yet to be developed is fruit picking perception. For picking strawberries, a robot needs to infer the location of picking points from the images of strawberries. Moreover, the size and weight of strawberries to be picked can help the robot to place the picked strawberries in proper punnets directly to be delivered to customers in supermarkets. This can save significant time and packing costs in packhouses. Geometry-based approaches are the most common approach to determine the picking point but they suffer from inaccuracies due to noise, occlusion, and varying shape and orientation of the berries. In contrast, we present two novel datasets of strawberries annotated with picking points, key-points (such as the shoulder points, the contact point between the calyx and flesh, and the point on the flesh farthest from the calyx), and the weight and size of the berries. We performed experiments with Detectron-2, which is an extended version of Mask-RCNN with key-points detection capability. The results show that the key-points detection approach works well for picking and grasping point localization. The second dataset also presents the dimensions and weight of strawberries. Our novel baseline model for weight estimation outperforms many state-of-the-art deep networks. The datasets and annotations are available at https://github.com/imanlab/strawberry-pp-w-r-dataset.

This paper presents a novel probabilistic approach to deep robot learning from demonstrations (LfD). Deep move-ment primitives (DMPs) are deterministic LfD model that maps visual information directly into a robot trajectory. This paper extends DMPs and presents a deep probabilistic model that maps the visual information into a distribution of effective robot trajectories. The architecture that leads to the highest level of trajectory accuracy is presented and compared with the existing methods. Moreover, this paper introduces a novel training method for learning domain-specific latent features. We show the superiority of the proposed probabilistic approach and novel latent space learning in the real-robot task of strawberry harvesting in the lab. The experimental results demonstrate that latent space learning can significantly improve model prediction performances. The proposed approach allows to sample trajectories from distribution and optimises the robot trajectory to meet a secondary objective, e.g. collision avoidance.