Unsupervised semantic segmentation aims to discover groupings within images, capturing objects' view-invariance without external supervision. Moreover, this task is inherently ambiguous due to the varying levels of semantic granularity. Existing methods often bypass this ambiguity using dataset-specific priors. In our research, we address this ambiguity head-on and provide a universal tool for pixel-level semantic parsing of images guided by the latent representations encoded in self-supervised models. We introduce a novel algebraic approach that recursively decomposes an image into nested subgraphs, dynamically estimating their count and ensuring clear separation. The innovative approach identifies scene-specific primitives and constructs a hierarchy-agnostic tree of semantic regions from the image pixels. The model captures fine and coarse semantic details, producing a nuanced and unbiased segmentation. We present a new metric for estimating the quality of the semantic segmentation of discovered elements on different levels of the hierarchy. The metric validates the intrinsic nature of the compositional relations among parts, objects, and scenes in a hierarchy-agnostic domain. Our results prove the power of this methodology, uncovering semantic regions without prior definitions and scaling effectively across various datasets. This robust framework for unsupervised image segmentation proves more accurate semantic hierarchical relationships between scene elements than traditional algorithms. The experiments underscore its potential for broad applicability in image analysis tasks, showcasing its ability to deliver a detailed and unbiased segmentation that surpasses existing unsupervised methods.

Cloud removal is a relevant topic in Remote Sensing, fostering medium- and high-resolution optical image usability for Earth monitoring and study. Recent applications of deep generative models and sequence-to-sequence-based models have proved their capability to advance the field significantly. Nevertheless, there are still some gaps: the amount of cloud coverage, the landscape temporal changes, and the density and thickness of clouds need further investigation. We fill some of these gaps in this work by introducing an innovative deep model. The proposed model is multi-modal, relying on both spatial and temporal sources of information to restore the whole optical scene of interest. We use the outcomes of both temporal-sequence blending and direct translation from Synthetic Aperture Radar (SAR) to optical images to obtain a pixel-wise restoration of the whole scene. The reconstructed images preserve scene details without resorting to a considerable portion of a clean image. Our approach’s advantage is demonstrated across various atmospheric conditions tested on different datasets. Quantitative and qualitative results prove that the proposed method obtains cloud-free images coping with landscape changes.

Robots assisting humans with some specific tasks have been demonstrated on several occasions. A further challenging idea is to anticipate human needs by mining the future demand from the next action prediction. To trigger this anticipation mechanism a robot has to recognize what the human is doing now, foresee what the human will do next, and from their connection guesstimating what to do to help. We propose here a deep network combining the essential components of this challenging process leading to foreseeing the help that can be provided in human-robot collaboration.

A methodology for 3D surface modeling from a single image is proposed. The principal novelty is concave and specular surface modeling without any externally imposed prior. The main idea of the method is to use BRDFs and generated rendered surfaces, to transfer the normal field, computed for the generated samples, to the unknown surface. The transferred information is adequate to blow and sculpt the segmented image mask in to a bas-relief of the object. The object surface is further refined basing on a photo-consistency formulation that relates for error minimization the original image and the modeled object.

We present a method for estimating the contact event between sensor-free active subtracks, named flippers, of an articulated tracked vehicle (ATV) and the terrain surface. The main idea is to consider both the moving base link and unexpected collisions dynamics as disturbances of the flipper dynamics. On this basis we extend the generalized momenta fault detection and isolation (FDI) method to compute the residual dynamics of the flippers, without resorting to additional sensory information. Under the hypothesis that the residual signal presents disturbance patterns that can be discriminated by those generated by unexpected collisions of the flippers with the ground, we apply a classification method to recover the contact event. The wavelet packet transform is used to decompose the signal and to generate a feature space for the residual, from the different subbands. Finally, sparse SVM, based on feature selection discriminates the contact signal.

We propose a novel approach to human action recognition, with motion capture data (MoCap), based on grouping sub-body parts. By representing configurations of actions as manifolds, joint positions are mapped on a subspace via principal geodesic analysis. The reduced space is still highly informative and allows for classification based on a non-parametric Bayesian approach, generating behaviors for each sub-body part. Having partitioned the set of joints, poses relative to a sub-body part are exchangeable, given a specified prior and can elicit, in principle, infinite behaviors. The generation of these behaviors is specified by a Dirichlet process mixture. We show with several experiments that the recognition gives very promising results, outperforming methods requiring temporal alignment.

We introduce a novel framework for modeling articulated objects based on the aspects of their components. By decomposing the object into components, we divide the problem in smaller modeling tasks. After obtaining 3D models for each component aspect by employing a shape deformation paradigm, we merge them together, forming the object components. The final model is obtained by assembling the components using an optimization scheme which fits the respective 3D models to the corresponding apparent contours in a reference pose. The results suggest that our approach can produce realistic 3D models of articulated objects in reasonable time.

We present a real time method for updating a 3D map with dynamic obstacles detection. Moving obstacles are detected through ray-casting on spherical voxelization of point clouds. We evaluate the accuracy of this method on a point cloud dataset, suitably constructed for testing ray-surface intersection under relative motion conditions. Moreover, we show the benefits of the map updating in a real robot equipped with a rotating LIDAR system, navigating in real world scenarios, populated by moving people.

We propose in this article a new approach to robot cognitive control based on a stimulus-response framework that models both a robot’s stimuli and the robot’s decision to switch tasks in response to or inhibit the stimuli. In an autonomous system, we expect a robot to be able to deal with the whole system of stimuli and to use them to regulate its behavior in real-world applications. The proposed framework contributes to the state of the art of robot planning and high-level control in that it provides a novel perspective on the interaction between robot and environment. Our approach is inspired by Gibson’s constructive view of the concept of a stimulus and by the cognitive control paradigm of task switching. We model the robot’s response to a stimulus in three stages. We start by defining the stimuli as perceptual functions yielded by the active robot processes and learned via an informed logistic regression. Then we model the stimulus-response relationship by estimating a score matrix that leads to the selection of a single response task for each stimulus, basing the estimation on low-rank matrix factorization. The decision about switching takes into account both an interference cost and a reconfiguration cost. The interference cost weighs the effort of discontinuing the current robot mental state to switch to a new state, whereas the reconfiguration cost weighs the effort of activating the response task. A choice is finally made based on the payoff of switching. Because processes play such a crucial role both in the stimulus model and in the stimulus-response model, and because processes are activated by actions, we address also the process model, which is built on a theory of action. The framework is validated by several experiments that exploit a full implementation on an advanced robotic platform and is compared with two known approaches to replanning. Results demonstrate the practical value of the system in terms of robot autonomy, flexibility, and usability.

Attention is a concept of human perception that enables human subjects to select the potentially relevant parts out of the huge amount of sensory data and that enables interactions with other human subjects by sharing attention with each other. These abilities are also of large interest for autonomous robots, therefore, interest in modeling concepts of human attention computationally has increased strongly in the robotics community during the last decade. Especially in human-robot interaction, the ability to detect what a human partner is attending to and to act in a similar way to enable intuitive communication, are important skills for a robotic system.

The paper presents a novel framework for 3D autonomous navigation for tracked vehicles. The framework takes care of clustering and segmentation of point clouds, traversability analysis, autonomous 3D path planning, motion planning and flippers control. Results illustrated in an experiment section show that the framework is promising to face harsh terrains. Robot performance is proved in three main experiments taken in a training rescue area, on fire escape stairs and in a non-planar testing environment, built ad-hoc to prove 3D path planning functionalities. Performance tests are also presented.

Modeling cognitive control is a major issue in robot control, and it is about deciding when a task cannot succeed and a new task need to be initiated. These decisions are induced by incoming stimuli alerting of events taking place while the robot is executing its duties. To learn cognitive control we address the human inspired mechanisms that govern cognitive control and that have been widely studied in neuroscience, namely, shifting and inhibition. Shifting and inhibition are, in fact, executive cognitive functions responding selectively to stimuli, so as to switch from one activity to a more compelling one or to inhibit inappropriate urges and preserve focus on the current task. In an autonomous system these cognitive skills are crucial to assess a well-regulated reactive behavior, which is of particular relevance in critical circumstances. In this paper we illustrate a new method developed for learning shifting and inhibition, based on Gaussian Processes, and using examples provided by skilled operators. We finally show that the learning method is promising and can be seen as a new view for modeling robot reactive and proactive behaviors.

A novel approach to 3D gaze estimation for wearable multi-camera devices is proposed and its effectiveness is demonstrated both theoretically and empirically. The proposed approach, firmly grounded on the geometry of the multiple views, introduces a calibration procedure that is efficient, accurate, highly innovative but also practical and easy. Thus, it can run online with little intervention from the user. The overall gaze estimation model is general, as no particular complex model of the human eye is assumed in this work. This is made possible by a novel approach, that can be sketched as follows: each eye is imaged by a camera; two conics are fitted to the imaged pupils and a calibration sequence, consisting in the subject gazing a known 3D point, while moving his/her head, provides information to 1) estimate the optical axis in 3D world; 2) compute the geometry of the multi-camera system; 3) estimate the Point of Regard in 3D world. The resultant model is being used effectively to study visual attention by means of gaze estimation experiments, involving people performing natural tasks in wide-field, unstructured scenarios.

In this paper we present an integrated approach to COSMO-SkyMed image analysis and classification exploiting integration of different data of the regions of interest, namely urban forestry areas, wide urban parks. The aim is to provide a methodology for exploiting complex data structures built upon multi resolution grids gathering together with optical and X-SAR images, also historical land exploitation and meteorological data, records of human habits, and several other information sources. Although these data are specifically gathered to built a fire susceptibility map, the method is quite general. Indeed, the contribution of the model and its novelty relies manly on the definition of a learning schema lifting different factors and aspects of the event to be identified (here fire causes), including physical, social and behavioral ones, to the design of a fire susceptibility map, of a specific urban forestry. The outcome is an integrated geospatial database providing an infrastructure that merges cartography, heterogeneous data and complex analysis, in so establishing a digital environment where users and tools are interactively connected in an efficient and flexible way.

The ability to follow the gaze of conspecifics is a critical component in the development of social behaviors, and many efforts have been directed to studying the earliest age at which it begins to develop in infants. Developmental and neurophysiological studies suggest that imitative learning takes place once gaze-following abilities are fully established and joint attention can support the shared behavior required by imitation. Accordingly, gaze-following acquisition should be precursory to most machine learning tasks, and imitation learning can be seen as the earliest modality for acquiring meaningful gaze shifts and for understanding the structural substrate of fixations. Indeed, if some early attentional process, based on a suitable combination of gaze shifts and fixations, could be learned by the robot, then several demonstration learning tasks would be dramatically simplified. In this paper, we describe a methodology for learning gaze shifts based on imitation of gaze following with a gaze machine, which we purposefully introduced to make the robot gaze imitation conspicuous. The machine allows the robot to share and imitate gaze shifts and fixations of a caregiver through a mutual vergence. This process is then suitably generalized by learning both the scene salient features toward which the gaze is directed and the way saccadic programming is attained. Salient features are modeled by a family of Gaussian mixtures. These together with learned transitions are generalized via hidden Markov models to account for humanlike gaze shifts allowing to discriminate salient locations.

Visual attention is becoming an increasingly imperative capability to endow computer vision systems and autonomous agents with. Starting from a biological inspired model of attention, we present an experiment aimed to study selective attention in 3D space. Depth has been proved to be an important feature affecting the way attention is deployed when observing a scene. We studied preferential scanning paths and fixation zones in a task-driven wandering of the tutor gaze over a scene where multiple targets had been disposed on different depth planes. We supposed that selective attention would aggregate targets in cliques that maximize utility, minimizing meanwhile visual effort produced when passing from closer planes to further planes or between different cluttered locations. By means of a purposely designed machine, we stored visual and motor data of the tutor's head; we clustered different scanning paths of the gaze shifts according to velocity and space criteria to determine a preference model of attentional shifts and fixations. We propose subsequently a utility model that can formalize acquired information and establish a vision-based attentional framework for robots. We show that an interpretation of task-driven gaze orienting based on the presented preference criteria correctly accounts for the studied behaviours, as further reported in the literature

In this work we describe a model-based approach to the executive control of a rescue rover. We show how this control architecture naturally supports human-robot interaction in the diverse activities needed in rescue and search. We illustrate the approach by considering human-robot interaction in the domain of the RoboCup rescue competition. We discuss the implementation and tests done both during RoboCup contests and in the laboratory, to show performances according to different working modalities such as fully operated, supervised, fully autonomous.

We focus on a rich axiomatization for actions in the situation calculus that includes, among other features, a solution to the frame problem for deterministic actions. Our work is foundational in nature, directed at simplifying the entailment problem for these axioms. Specifically, we make four contributions to the metatheory of situation calculus axiomatizations of dynamical systems: (1) We prove that the above-mentioned axiomatization for actions has a relative satisfiability property; the full axiomatization is satisfiable iff the axioms for the initial state are. (2)We define the concept of regression relative to these axioms, and prove a soundness and completeness theorem for a regression-based approach to the entailment problem for a wide class of queries. (3) Our formalization of the situation calculus requires certain foundational axioms specifying the domain of situations. These include an induction axiom, whose presence complicates human and automated reasoning in the situation calculus. We characterize various classes of sentences whose proofs do not require induction, and in some cases, some of the other foundational axioms. (4)We prove that the logic programming language GOLOG never requires any of the foundational axioms for the evaluation of programs.

We present a general proof theoretical methodology for default systems. Given a default theory W, D, the default rules D are simply understood as restrictions on the tableaux construction of the logic. Different default approaches approch (such as Reiter, Brewka or Lukaszewicz), the allowable default extensions can be obtained from the default tableau construction. The advantage of our approach, besides being simple and neat, is in its generality: it allows for the development of a default theory for any logic with a tableau formulation, such as intuitionistic logic, linear logic or modal logic.