Technologies to enable safe and effective collaboration and coexistence between humans and robots have gained significant importance in the last few years. A critical component useful for realizing this collaborative paradigm is the understanding of human and robot 3D poses using non-invasive systems. Therefore, in this paper, we propose a novel vision-based system leveraging depth data to accurately establish the 3D locations of skeleton joints. Specifically, we introduce the concept of Pose Nowcasting, denoting the capability of the proposed system to enhance its current pose estimation accuracy by jointly learning to forecast future poses. The experimental evaluation is conducted on two different datasets, providing accurate and real-time performance and confirming the validity of the proposed method on both the robotic and human scenarios. • We introduce the novel task of 3D Pose Nowcasting. • Our Pose Nowcasting system is based on both 3D Pose Estimation and Forecasting. • We show that knowledge about pose forecasting improves the accuracy of pose estimation. • We apply the proposed system both to human and robots. • Result on different dataset show state-of-the-art performance and robustness.

Artificial intelligence (AI)-native edge networks are promising solutions to seamlessly integrate AI into modern network architectures, promoting intelligent and hyper-flexible behavior in self-adaptation and reconfiguration of hybrid networks. AI-native edge nodes have to promptly react to any change in network conditions and to be linked to Internet of Everything (IoE) deployed in etherogeneous communication domains. This paper deals with a next-generation programmable edge node capable of being employed in different domains in a unified and flexible manner. With the aim to achieve a low re-configuration overhead in such herogeneous contexts, this paper proposes the integration of a generative-AI module within an edge node exploiting foundation models for efficient general-purpose time-series prediction, without involving overhead and costs due to models trained from scratch and overcoming data scarcity. This permits to manage IoE networks deployed in both homogeneous and heterogeneous domains, i.e., aqua, ground and air, autonomously, without the need for adjustments from the outside. As foundation models, we focused on Chronos and TimesFM, in both the zero-shot and fine-tuning learning paradigms. Finally, performance results in terms of prediction accuracy, training, and inference time are provided and compared with those achieved by the state-of-the-art recurrent neural networks trained from scratch and baseline alternatives. The obtained results corroborate the potential of foundation models as key enablers of native AI networks, achieving good accuracy despite the absence of a training phase (i.e., in zero-shot mode) or with limited training (fine-tuning), w.r.t. alternatives.

This manuscript presents an improved version of the Inclusive Genetic Programming (IGP) algorithm. The IGP was developed to promote and maintain the population's genotypic diversity and showed superior performance compared to standard Genetic Programming (GP). In this work, two modifications to the IGP are proposed: first, the diversity promotion and maintenance mechanism is enhanced with information from the phenotype of the individuals rather than only the genotype; second, the Evolutionary Demes Despeciation Algorithm - V2 (EDDA-V2) is used to initialize the population. The phenotype is considered to differentiate the individuals also according to their behaviour rather than only their structure, while EDDA-V2 is employed to start the evolution with a simultaneously diverse and fit population, contrary to traditional initialization techniques. The algorithms incorporating these improvements are called Full Inclusive Genetic Programming (FIGP) and FIGP _E, respectively with and without the EDDA-V2 initialization. The experimental results, performed over eight benchmarks and considering six algorithms, demonstrate the superior performance of FIGP and FIGP _E in comparison to other GP formulations. Moreover, the EDDA-V2 initialization allows for a significant reduction of the computational time.

Effective modeling of human interactions is of utmost importance when forecasting behaviors such as future trajectories. Each individual, with its motion, influences surrounding agents since everyone obeys to social non-written rules such as collision avoidance or group following. In this paper we model such interactions, which constantly evolve through time, by looking at the problem from an algorithmic point of view, i.e., as a data manipulation task. We present a neural network based on an end-to-end trainable working memory, which acts as an external storage where information about each agent can be continuously written, updated and recalled. We show that our method is capable of learning explainable cause-effect relationships between motions of different agents, obtaining state-of-the-art results on multiple trajectory forecasting datasets.

Prediction of head movements in immersive media is key to designing efficient streaming systems able to focus the bandwidth budget on visible areas of the content. However, most of the numerous proposals made to predict user head motion in 360° images and videos do not explicitly consider a prominent characteristic of the head motion data: its intrinsic uncertainty. In this article, we present an approach to generate multiple plausible futures of head motion in 360° videos, given a common past trajectory. To our knowledge, this is the first work that considers the problem of multiple head motion prediction for 360° video streaming. We introduce our discrete variational multiple sequence (DVMS) learning framework, which builds on deep latent variable models. We design a training procedure to obtain a flexible, lightweight stochastic prediction model compatible with sequence-to-sequence neural architectures. Experimental results on four different datasets show that DVMS outperforms competitors adapted from the self-driving domain by up to 41% on prediction horizons up to 5 s, at lower computational and memory costs. To understand how the learned features account for the motion uncertainty, we analyze the structure of the learned latent space and connect it with the physical properties of the trajectories. We also introduce a method to estimate the likelihood of each generated trajectory, enabling the integration of DVMS in a streaming system. We hence deploy an extensive evaluation of the interest of our DVMS proposal for a streaming system. To do so, we first introduce a new Python-based 360° streaming simulator that we make available to the community. On real-world user, video, and networking data, we show that predicting multiple trajectories yields higher fairness between the traces, the gains for 20–30% of the users reaching up to 10% in visual quality for the best number K of trajectories to generate.

Pedestrians and drivers are expected to safely navigate complex urban environments along with several non cooperating agents. Autonomous vehicles will soon replicate this capability. Each agent acquires a representation of the world from an egocentric perspective and must make decisions ensuring safety for itself and others. This requires to predict motion patterns of observed agents for a far enough future. In this paper we propose MANTRA, a model that exploits memory augmented networks to effectively predict multiple trajectories of other agents, observed from an egocentric perspective. Our model stores observations in memory and uses trained controllers to write meaningful pattern encodings and read trajectories that are most likely to occur in future. We show that our method is able to natively perform multi-modal trajectory prediction obtaining state-of-the art results on four datasets. Moreover, thanks to the non-parametric nature of the memory module, we show how once trained our system can continuously improve by ingesting novel patterns.

Prediction of head movements in immersive media is key to design efficient streaming systems able to focus the bandwidth budget on visible areas of the content. Numerous proposals have therefore been made in the recent years to predict 360° images and videos. However, the performance of these models is limited by a main characteristic of the head motion data: its intrinsic uncertainty. In this article, we present an approach to generate multiple plausible futures of head motion in 360° videos, given a common past trajectory. Our method provides likelihood estimates of every predicted trajectory, enabling direct integration in streaming optimization. To the best of our knowledge, this is the first work that considers the problem of multiple head motion prediction for 360° video streaming. We first quantify this uncertainty from the data. We then introduce our discrete variational multiple sequence (DVMS) learning framework, which builds on deep latent variable models. We design a training procedure to obtain a flexible and lightweight stochastic prediction model compatible with sequence-to-sequence recurrent neural architectures. Experimental results on 3 different datasets show that our method DVMS outperforms competitors adapted from the self-driving domain by up to 37% on prediction horizons up to 5 sec., at lower computational and memory costs. Finally, we design a method to estimate the respective likelihoods of the multiple predicted trajectories, by exploiting the stationarity of the distribution of the prediction error over the latent space. Experimental results on 3 datasets show the quality of these estimates, and how they depend on the video category.

Assessing the similarity of two images is a complex task that attracts significant efforts in the image processing community. The widely used structural similarity index measure (SSIM) addresses this problem by quantifying a perceptual structural similarity. In this paper we consider a recently introduced continuous SSIM (cSSIM), which allows one to analyze sequences of images of increasingly fine resolutions, and further extend the definition of the index to encompass the locally weighted version that is used in practice. For both the local and the global versions, we prove that the continuous index includes the classical SSIM as a special case, and we provide a precise connection between image similarity measured by the cSSIM and by the $L_2$ norm. Using this connection, we derive bounds on the cSSIM by means of bounds on the $L_2$ error, and we even prove that the two error measures are equivalent in certain circumstances. We exploit these results to obtain precise rates of convergence with respect to the cSSIM for several concrete image interpolation methods, and we further validate these findings by different numerical experiments. This newly established connection paves the way to obtain novel insights into the features and limitations of the SSIM, including on the effect of the local weighted window on the index performances.

Autonomous vehicles are expected to drive in complex scenarios with several independent non cooperating agents. Path planning for safely navigating in such environments can not just rely on perceiving present location and motion of other agents. It requires instead to predict such variables in a far enough future. In this paper we address the problem of multimodal trajectory prediction exploiting a Memory Augmented Neural Network. Our method learns past and future trajectory embeddings using recurrent neural networks and exploits an associative external memory to store and retrieve such embeddings. Trajectory prediction is then performed by decoding in-memory future encodings conditioned with the observed past. We incorporate scene knowledge in the decoding state by learning a CNN on top of semantic scene maps. Memory growth is limited by learning a writing controller based on the predictive capability of existing embeddings. We show that our method is able to natively perform multi-modal trajectory prediction obtaining state-of-the art results on three datasets. Moreover, thanks to the non-parametric nature of the memory module, we show how once trained our system can continuously improve by ingesting novel patterns.

In this paper an initial approach to Intelligent Control (IC) using Genetic Programming (GP) for access to space applications is presented. GP can be employed successfully to design a controller even for complex systems, where classical controllers fail because of the high nonlinearity of the systems. The main property of GP, that is its ability to autonomously create explicit mathematical equations starting from a very poor knowledge of the considered plant, or just data, can be exploited for a vast range of applications. Here, GP has been used to design the control law in an Intelligent Control framework for a modified version of the Goddard Rocket problem in 3 different failure scenarios, where the approach to IC consists in an online re-evaluation of the control law using GP when a considerably big change in the environment or in the plant happens. The presented results are then used to highlight the potential benefits of the method, as well as aspects that will need further developments.