The increasing diffusion of deepfakes has raised significant global concerns, especially due to their potential geopolitical implications. This concern relates to the spread of false information that can mislead people and have a serious impact on societies. However, identifying what can misinform people is not trivial. In this work, we present an experimental study that involves a sample of students of different backgrounds. Three different political deepfakes were created and shown to them. The perceived values were then analyzed using a specific questionnaire. The experimental results show significant differences in the ability to discern the authenticity of the proposed videos depending on the level of awareness of the contents viewed. This demonstrates the crucial role of education on deepfakes in countering the spread of misinformation.

Large Language Models (LLMs) and Visual Language Models (VLMs) are attracting increasing interest due to their improving performance and applications across various domains and tasks. However, LLMs and VLMs can produce erroneous results, especially when a deep understanding of the problem domain is required. For instance, when planning and perception are needed simultaneously, these models often struggle because of difficulties in merging multi-modal information. To address this issue, fine-tuned models are typically employed and trained on specialized data structures representing the environment. This approach has limited effectiveness, as it can overly complicate the context for processing. In this paper, we propose a multi-agent architecture for embodied task planning that operates without the need for specific data structures as input. Instead, it uses a single image of the environment, handling free-form domains by leveraging commonsense knowledge. We also introduce a novel, fully automatic evaluation procedure, PG2S, designed to better assess the quality of a plan. We validated our approach using the widely recognized ALFRED dataset, comparing PG2S to the existing KAS metric to further evaluate the quality of the generated plans.

Task planning for robots in real-life settings presents significant challenges. These challenges stem from three primary issues: the difficulty in identifying grounded sequences of steps to achieve a goal; the lack of a standardized mapping between high-level actions and low-level commands; and the challenge of maintaining low computational overhead given the limited resources of robotic hardware. We introduce EMPOWER, a framework designed for open-vocabulary online grounding and planning for embodied agents aimed at addressing these issues. By leveraging efficient pre-trained foundation models and a multi-role mechanism, EMPOWER demonstrates notable improvements in grounded planning and execution. Quantitative results highlight the effectiveness of our approach, achieving an average success rate of 0.73 across six different real-life scenarios using a TIAGo robot.

Artificial Intelligence (AI) is an enabling technology in the context of Industry 4.0. In particular, the automotive sector is among those who can benefit most of the use of AI in conjunction with advanced vision techniques. The scope of this work is to integrate deep learning algorithms in an industrial scenario involving a robotic Peg-in-Hole task. More in detail, we focus on a scenario where a human operator manually positions a carbon fiber automotive part in the workspace of a 7 Degrees of Freedom (DOF) manipulator. To cope with the uncertainty on the relative position between the robot and the workpiece, we adopt a three stage strategy. The first stage concerns the Three-Dimensional (3D) reconstruction of the workpiece using a registration algorithm based on the Iterative Closest Point (ICP) paradigm. Such a procedure is integrated with a semantic image segmentation neural network, which is in charge of removing the background of the scene to improve the registration. The adoption of such network allows to reduce the registration time of about 28.8%. In the second stage, the reconstructed surface is compared with a Computer Aided Design (CAD) model of the workpiece to locate the holes and their axes. In this stage, the adoption of a Convolutional Neural Network (CNN) allows to improve the holes’ position estimation of about 57.3%. The third stage concerns the insertion of the peg by implementing a search phase to handle the remaining estimation errors. Also in this case, the use of the CNN reduces the search phase duration of about 71.3%. Quantitative experiments, including a comparison with a previous approach without both the segmentation network and the CNN, have been conducted in a realistic scenario. The results show the effectiveness of the proposed approach and how the integration of AI techniques improves the success rate from 84.5% to 99.0%.

Social robots can be used for assisting children managing chronic illness through education and encouragement. In this paper, we present a study about the use of a NAO robot in the therapy with children diagnosed with an autism spectrum disorder (ASD). In particular, we propose an approach to track the gaze of the child while she/he is interacting with the robot. We adopt a two level architecture, where the high-levels task in the treatment protocol are decided by the therapist and the robot performs autonomously the low-level tasks. We carried out a preliminary evaluation of the proposed approach involving neurotypical and an autistic children.

The success of the interaction between the robotics community and the users of these services is an aspect of considerable importance in the drafting of the development plan of any technology. This aspect becomes even more relevant when dealing with sensitive services and issues such as those related to interaction with specific subgroups of any population. Over the years, there have been few successes in integrating and proposing technologies related to deafness and sign language. Instead, in this paper, we propose an account of successful interaction between a signatory robot and the Italian deaf community, which occurred during the Smart City Robotics Challenge (SciRoc) 2021 competition1. Thanks to the use of a participatory design and the involvement of experts belonging to the deaf community from the early stages of the project, it was possible to create a technology that has achieved significant results in terms of acceptance by the community itself and could lead to significant results in the technology development as well.

This paper presents a method to cope with autonomous assembly tasks in the presence of uncertainties. To this aim, a Peg-in-Hole operation is considered, where the target workpiece position is unknown and the peg-hole clearance is small. Deep learning based hole detection and 3D surface reconstruction techniques are combined for accurate workpiece localization. In detail, the hole is detected by using a convolutional neural network (CNN), while the target workpiece surface is reconstructed via 3D-Digital Image Correlation (3D-DIC). Peg insertion is performed via admittance control that confers the suitable compliance to the peg. Experiments on a collaborative manipulator confirm that the proposed approach can be promising for achieving a better degree of autonomy for a class of robotic tasks in partially structured environments.

The use of embedded boards on robots, including unmanned aerial and ground vehicles, is increasing thanks to the availability of GPU equipped low-cost embedded boards in the market. Porting algorithms originally designed for desktop CPUs on those boards is not straightforward due to hardware limitations. In this paper, we present how we modified and customized the open source SLAM algorithm ORB-SLAM2 to run in real-time on the NVIDIA Jetson TX2. We adopted a data flow paradigm to process the images, obtaining an efficient CPU/GPU load distribution that results in a processing speed of about 30 frames per second. Quantitative experimental results on four different sequences of the KITTI datasets demonstrate the effectiveness of the proposed approach. The source code of our data flow ORB-SLAM2 algorithm is publicly available on GitHub.

While robots are more and more deployed among people in public spaces, the impact of cyber-security attacks is significantly increasing. Most of consumer and professional robotic systems are affected by multiple vulnerabilities and the research in this field is just started. This paper addresses the problem of automatic detection of anomalous behaviors possibly coming from cyber-security attacks. The proposed solution is based on extracting system logs from a set of internal variables of a robotic system, on transforming such data into images, and on training different Autoencoder architectures to classify robot behaviors to detect anomalies. Experimental results in two different scenarios (autonomous boats and social robots) show effectiveness and general applicability of the proposed method.

In this paper, we propose the use of subspace clustering to detect the states of dynamical systems from sequences of observations. In particular, we generate sparse and interpretable models that relate the states of aquatic drones involved in autonomous water monitoring to the properties (e.g., statistical distribution) of data collected by drone sensors. The subspace clustering algorithm used is called SubCMedians. A quantitative experimental analysis is performed to investigate the connections between i) learning parameters and performance, ii) noise in the data and performance. The clustering obtained with this analysis outperforms those generated by previous approaches.

Automatic surveillance systems for the maritime domain are becoming more and more important due to a constant increase of naval traffic and to the simultaneous reduction of crews on decks. However, available technology still provides only a limited support to this kind of applications. In this paper, a modular system for intelligent maritime surveillance, capable of fusing information from heterogeneous sources, is described. The system is designed to enhance the functions of the existing vessel traffic services systems and to be deployable in populated areas, where radar-based systems cannot be used due to the high electromagnetic radiation emissions. A quantitative evaluation of the proposed approach has been carried out on a large and publicly available data set of images and videos, which are collected from multiple real sites, with different light, weather, and traffic conditions.

Detection, classification, and tracking of people and vehicles are fundamental processes in intelligent surveillance systems. The use of publicly available data set is the appropriate way to compare the relative merits of existing methods and to develop and assess new robust solutions. In this paper, we focus on the maritime domain and we describe the generation of boat classification data sets, containing images of boats automatically extracted by the ARGOS system, operating 24/7 in Venice, Italy. The data sets are unique in their nature, since they come from an incomparable environment like Venice, but they present very interesting challenges to vehicle classification, due to changes in the environmental conditions, boat wakes, waves, reflections, etc. We thus believe that robust techniques, validated through the ARGOS Boat Classification data sets, will improve the development and deployment of solutions in similar applications related to vehicle detection and classification.

Background modeling in fast changing scenarios is a challenging task due to unexpected events like sudden illumination changes, reflections, and shadows, which can strongly affect the accuracy of the foreground detection. In this paper, we describe a real-time and effective background modeling approach, called FAFEX, that can deal with global and rapid changes in the scene background. The method is designed to identify variations in the background geometry of the monitored scene and it has been quantitatively tested on a publicly available data set, containing a varied set of highly dynamic environments. The experimental evaluation demonstrates how our method is able to effectively deals with challenging sequences in real-time.

The detection of malignant lesions in dermoscopic images by using automatic diagnostic tools can help in reducing mortality from melanoma. In this paper, we describe a fully-automatic algorithm for skin lesion segmentation in dermoscopic images. The proposed approach is highly accurate when dealing with benign lesions, while the detection accuracy significantly decreases when melanoma images are segmented. This particular behavior lead us to consider geometrical and color features extracted from the output of our algorithm for classifying melanoma images, achieving promising results.

We propose an adaptive tracking system for assisted living that integrates user information about emergency events. Information fusion between user data and visual data is performed in order to estimate and assess the situation at hand. The system is able to dynamically switch between different segmentation and tracking algorithms improving its performance, as shown by the proposed examples.

Visual surveillance in dynamic scenes is currently one of the most active research topics in computer vision, many existing applications are available. However, difficulties in realizing effective video surveillance systems that are robust to the many different conditions that arise in real environments, make the actual deployment of such systems very challenging. In this article, we present a real, unique and pioneer video surveillance system for boat traffic monitoring, ARGOS. The system runs continuously 24 hours a day, 7 days a week, day and night in the city of Venice (Italy) since 2007 and it is able to build a reliable background model of the water channel and to track the boats navigating the channel with good accuracy in real-time. A significant experimental evaluation, reported in this article, has been performed in order to assess the real performance of the system.