Large-scale evaluation of state-of-the-art visual SLAM has shown that its tracking performance degrades considerably if the camera view is not adjusted to avoid the low-texture areas. Deep reinforcement learning based approaches have been proposed to improve the robustness of visual tracking in such unsupervised settings. Our extensive analysis reveals the fundamental limitations of reinforcement learning based active view planning, especially in transition scenarios (entering/exiting the room, texture-less walls, and lobbies). In challenging transition scenarios, the agent generally remains unable to cross the transition during training, limiting its ability to learn the maneuver. We propose human-supervised RL training (imitation learning) and achieve significantly improved performance after$\sim$50 hours of supervised training. To reduce longer human supervision requirements, we also explore fine-tuning our network with an online learning policy. Here, we use limited human-supervised training ($\sim$20 hours), and fine-tune the network with unsupervised training ($\sim$45 hours), obtaining encouraging results. We also release our multi-model, human supervised training dataset. The dataset contains challenging and diverse transition scenarios and can aid the development of imitation learning policies for consistent visual tracking. We also release our implementation11https://tinyurl.com/478maz6u.

This article addresses the crucial aspect of safety in collaborative robotics by introducing a new continuous wavelet transform-convolutional neural network (CWT-CNN) for efficient robot collision detection. Unlike conventional methods, CWT-CNN exhibits superior data efficiency, requiring minimal collision data for robust training without relying on a dynamic model. The network’s adaptability extends to varying internal stiffness levels, offering robustness to changes in robotic system characteristics. Through comprehensive experimental studies, we investigate the impact of input signal types, wavelet types, wavelet scale ranges, and time-moving window sizes on collision detection performance, offering critical insights for optimal CWT parameter selection. Additionally, our transferability analysis demonstrates that the CWT-CNN can seamlessly adapt from one joint to another, requiring only minimal free-motion data from the new joint. This adaptability is validated through extensive experiments on an industrial robot and the robot equipped with variable stiffness actuators. In conclusion, the CWT-CNN is highly generalizable and data-efficient, making it a reliable solution for real-time collision detection in human-robot interactions, addressing a key aspect of safety in collaborative environments.

Human activity recognition is a crucial area of research that involves understanding human movements using computer and machine vision technology. Deep learning has emerged as a powerful tool for this task, with models such as Convolutional Neural Networks (CNNs) and Transformers being employed to capture various aspects of human motion. One of the key contributions of this work is the demonstration of the effectiveness of feature fusion in improving human activity recognition accuracy, which has important implications for the development of more accurate and robust activity recognition systems. This approach addresses a limitation in the field, where the performance of existing models is often limited by their inability to capture both spatial and temporal features effectively. This work presents an approach for human activity recognition using sensory data extracted from four distinct datasets: HuGaDB, PKU-MMD, LARa, and TUG. Two models, the Parameter-Optimized Multi-Stage Graph Convolutional Network (PO-MS-GCN) and a Transformer, were trained and evaluated on each dataset to calculate accuracy and F1-score. Subsequently, the features from the last layer of each model were combined and fed into a classifier. The findings prove that PO MS-GCN outperforms state-of-the-art models in human activity recognition. Specifically, HuGaDB achieved an accuracy of 92.7% and f1-score of 95.2%, TUG achieved an accuracy of 93.2% and f1-score of 98.3%, while LARa and PKU-MMD achieved lower accuracies of 64.31% and 69%, respectively, with corresponding f1-scores of 40.63% and 48.16%. Moreover, feature fusion exceeded the PO-MS-GCN’s results in PKU-MMD, LARa, and TUG datasets.

Recent advancements in artificial intelligence algorithms for medical imaging show significant potential in automating the detection of lung infections from chest radiograph scans. However, current approaches often focus solely on either 2-D or 3-D scans, failing to leverage the combined advantages of both modalities. Moreover, conventional slice-based methods place a manual burden on radiologists for slice selection. To overcome these challenges, we propose the Recurrent 3-D Multi-level Vision Transformer (R3DM-ViT) model, capable of handling multimodal data to enhance diagnostic accuracy. Our quantitative evaluations demonstrate that R3DM-ViT surpasses existing methods, achieving an impressive accuracy of $96.67 \%$, F1-score of $96.88 \%$, mean average precision of $96.75 \%$, and mean average recall of $97.02 \%$. This research signifies a significant stride forward in the automated detection of lung infections through multimodal imaging.

Underwater robotic vision encounters numerous challenges posed by complex environments and varying lighting conditions. Meeting these challenges requires solutions that not only deliver accuracy but also demonstrate adaptability. In this paper, we present MARS (Multi-Scale Adaptive Robotics Vision), a pioneering approach to underwater object detection that prioritizes real-time performance and resource efficiency-essential attributes for underwater robotic systems. Leveraging a well-established object detection architecture, MARS integrates Domain-Adaptive Multi-Scale Attention (DAMSA), enhancing both detection accuracy and adaptability to diverse underwater domains. During training, DAMSA employs domain class-based attention, allowing the model to learn and prioritize features specific to different underwater environments. Extensive evaluation across diverse underwater datasets underscores the effectiveness of MARS. On the original dataset, MARS achieves an impressive mean Average Precision (mAP) of58.57%, demonstrating its proficiency in detecting critical underwater objects such as echinus, starfish, holothurian, scallop, and waterweeds. This capability positions MARS as a promising solution for applications in marine robotics, marine biology research, and environmental monitoring. Moreover, MARS exhibits exceptional resilience to domain shifts. When evaluated on an augmented dataset incorporating various enhancements, MARS delivers a commendable mAP of 36.16%, showcasing its robustness and adaptability in recognizing objects across diverse underwater conditions. The source code for MARS is publicly available on GitHub at https://github.com/LyesSaadSaoud/MARS-Object-Detection/.

The utilization of extra-robotic limbs to regain and extend motor abilities poses challenges. Nonetheless, the evidence suggesting the extension of these abilities without functional interference holds promise for the advancement of extra-robotic fingers or limbs. To gain insights into design improvement from diverse perspectives, we conducted a focus group study utilizing a prototype supernumerary robotic finger (SRF). The main purpose of the study is to gain knowledge on the current prototype, and then use the outcomes in a future co-designing process of SRF having the potential to be used as an assistive and augmented tool. Fifteen healthy subjects with an engineering background participated in the current study. We analyzed the collected data through a thematic analysis process to generate key design recommendations. These recommendations contain suggestions for designing for personal and social acceptance, adaptability, and multi-modal interaction. The results serve as a subset of future subsequent studies to be carried out with other stakeholders of the prototype, including patients, clinicians, and designers. Future work focuses on using the recommendations in a co-design process to inform the prototyping of SRF technologies supporting and extending motor abilities in upper limbs.

This research addresses the pressing challenge of weed identification in agriculture, crucial for ensuring food security in anticipation of a global population exceeding 9.7 billion by 2050. Utilizing drone imagery, we collected a dataset and proposed a customized model to achieve optimal performance. Our proposed model uses strategically modified backbone, neck, and head components, leveraging elements such as Ghost Convolution, BottleNeckCSP, and ECA (Efficient Channel Attention) layers. These modifications enhance the model’s capability to discern intricate patterns in drone imagery, ultimately leading to improved precision in weed detection. We introduce a purposefully crafted dataset to complement the model’s training, and our experiments demonstrate superior performance compared to the baseline models. Our model achieves a precision of 72.5%, recall of 68.0%, and [email protected] of 73.9, showcasing the effectiveness of our approach in enhancing detection accuracy. Leveraging a unique blend of feature extraction mechanisms, our model achieves remarkable accuracy in real-time soybean detection, outperforming established models like RT-DETR (Real-Time DEtection TransfoRmer) and YOLOv10. A detailed ablation study and comparative analysis with different YOLO versions and the transformer-based RT-DETR showcase the effectiveness of the proposed enhancements. Our work signifies a significant step towards advancing the field of precision agriculture, offering a model that is not only adaptive but also robust in identifying and localizing weeds in soybean fields.

Visual perception is an important component for autonomous navigation of unmanned surface vessels (USV), particularly for the tasks related to autonomous inspection and tracking. These tasks involve vision-based navigation techniques to identify the target for navigation. Reduced visibility under extreme weather conditions in marine environments makes it difficult for vision-based approaches to work properly. To overcome these issues, this paper presents an autonomous vision-based navigation framework for tracking target objects in extreme marine conditions. The proposed framework consists of an integrated perception pipeline that uses a generative adversarial network (GAN) to remove noise and highlight the object features before passing them to the object detector (i.e., YOLOv5). The detected visual features are then used by the USV to track the target. The proposed framework has been thoroughly tested in simulation under extremely reduced visibility due to sandstorms and fog. The results are compared with state-of-the-art de-hazing methods across the benchmarked MBZIRC simulation dataset, on which the proposed scheme has outperformed the existing methods across various metrics.

Loss of upper extremity motor control and function is an unremitting symptom in post-stroke patients. This would impose hardships on accomplishing their daily life activities. Supernumerary robotic limbs (SRLs) were introduced as a solution to regain the lost Degrees of Freedom (DoFs) by introducing an independent new limb. The actuation systems in SRL can be categorized into rigid and soft actuators. Soft actuators have proven advantageous over their rigid counterparts through intrinsic safety, cost, and energy efficiency. However, they suffer from low stiffness, which jeopardizes their accuracy. Variable Stiffness Actuators (VSAs) are newly developed technologies that have been proven to ensure accuracy and safety. In this paper, we introduce the novel Supernumerary Robotic Limb based on Variable Stiffness Actuators. Based on our knowledge, the proposed proof-of-concept SRL is the first that utilizes Variable Stiffness Actuators. The developed SRL would assist post-stroke patients in bi-manual tasks, e.g., eating with a fork and knife. The modeling, design, and realization of the system are illustrated. The proposed SRL was evaluated and verified for its accuracy via predefined trajectories. The safety was verified by utilizing the momentum observer for collision detection, and several post-collision reaction strategies were evaluated through the Soft Tissue Injury Test. The assistance process is qualitatively verified through standard user-satisfaction questionnaire.

The Fin-Ray principle, inspired by the physiology of fish rays, represents the foundation of a large number of robotic devices. However, despite their popularity, there is not anyad-hoctheoretical model technique for the analysis of this family of fingers. This lack is the main motivation of the presented work, which provides the mathematical modeling, analysis, and prototyping of a closed-chain Fin-Ray finger. In this scenario, the contribution of this article is twofold. At one end, we provide a general discrete Cosserat approach for the modeling of closed-chain soft robots which shares the geometrical structure of the rigid robotics counterpart. On the other end, the approach is employed to explore the family of Fin-Ray effect fingers. Finally, an improved design, which is able to conform to contacting surfaces, while maintaining stiffness out of its grasping plane, is fabricated and its performances are compared to those of a previously proposed prototype.

Flagellated micro-organism are regarded as excellent swimmers within their size scales. This, along with the simplicity of their actuation and the richness of their dynamics makes them a valuable source of inspiration to design continuum, self-propelled underwater robots. Here we introduce a soft, flagellum-inspired system which exploits the compliance of its own body to passively attain a range of geometrical configurations from the interaction with the surrounding fluid. The spontaneous formation of stable helical waves along the length of the flagellum is responsible for the generation of positive net thrust. We investigate the relationship between actuation frequency and material elasticity in determining the steady-state configuration of the system and its thrust output. This is ultimately used to perform a parameter identification procedure of an elastodynamic model aimed at investigating the scaling laws in the propulsion of flagellated robots.

In this paper, we introduce a set of guidelines for the design of grippers suitable for a safe human robot/interaction in cooperative tasks. Modularity, adaptability, robustness, intuitive control, limited weight are some of the key elements that could allow to effectively spread these devices in industrial and service applications. Following such guidelines, we present the prototype of the Co-Gripper: a robotic device for cooperative manipulation tasks with humans. The gripper is composed of two pairs of fingers, actuated with two motors, that can be controlled in a coordinated way or independently. Each finger has a modular underactuated structure, composed of three phalanges connected by passive joints. The gripper is wireless, so it can be easily connected both to the robotic arms and on passive structures. We designed a wearable wireless control interface composed of a ring and a bracelet allowing a simple and intuitive activation of the gripper without limiting human operator's manipulation capabilities. We performed a set of tests to quantify gripper performance and to exploit its potentialities in human-robot cooperation tasks.

In this work, we propose a method to compute the stiffness of flexible joints and its realization in order to let the fingers track a certain predefined trajectory. We refer to tendon-driven, underactuated and passively compliant hands composed of deformable joints and rigid links. Specific stiffness and pre-form shapes can be assigned to the finger joints can be given s such that a single-cable actuation can be used. We firstly define a procedure to determine suitable joints stiffness and then we propose a possible realization of soft joints using rapid prototyping techniques. The stiffness computation is obtained leveraging on the the mechanics of tendon-driven hands and on compliant systems, while for its implementation beam theory has been exploited. We validate the proposed framework both in simulation and with experiments using the robotic Soft-SixthFinger, a wearable robot for grasping compensation in patients with a paretic hand, as a case study. The proposed framework can be used to design the stiffness of the passive joints in several model of underactuated tendon-driven soft hands so to improve their grasping capabilities.

Wearable robots have been mostly designed as exoskeletons, with segments and joints corresponding to those of the person they are coupled with. Exoskeletons are mainly employed to augment human body force and precision capabilities, or for rehabilitation purposes. More recently, new wearable robots resembling additional robotic limbs have been developed thanks to the progress in miniaturization and efficiency of mechanical and sensing components. However, wearable robotic extra limbs presented in the literature lack of effective haptic feedback systems. In this paper, we present a robotic extra finger coupled with a vibrotactile ring interface. The human user is able to control the motion of the robotic finger through a switch placed on the ring, while being provided with vibrotactile feedback about the forces exerted by the robotic finger on the environment. To understand how to control the vibrotactile interface to evoke the most effective cutaneous sensations, we executed perceptual experiments to evaluate its absolute and differential thresholds. We also carried out a pick-and-place experiment with ten subjects. Haptic feedback significantly improved the performance in task execution in terms of completion time, exerted force, and perceived effectiveness. All subjects preferred experimental conditions employing haptic feedback with respect to those not providing any force feedback.

Robotic prosthesis are usually intended as artificial device extensions replacing a missing part of a human body. A new approach regarding robotic limbs is presented here. A modular robot is used not only for replacing a missing part of the body but also as an extra-limb in order to enhance manipulation dexterity and enlarge the workspace of human beings. In this work, the model and control of an additional finger, the Sixth-Finger, is presented as a case study of this type of robotic limbs. The robotic finger has been placed on the wrist opposite to the hand palm. This solution allows to enlarge the hand workspace, increasing the grasp capability of the user. An object-based mapping algorithm is proposed to control the robotic extra-finger by interpreting the whole hand motion in grasping action. A four DoFs modular prototype is presented along with numerical simulations and real experiments. The proposed Sixth-Finger can lead to a wide range of applications in the direction of augmenting human capabilities through wearable robotics.