This paper introduces the Proactive Assistance through action-Completion Estimation (PACE) framework, designed to enhance human-robot collaboration through real-time monitoring of human progress. PACE incorporates a novel method that combines Dynamic Time Warping (DTW) with correlation analysis to track human task progression from hand movements. PACE trains a reinforcement learning policy from limited demonstrations to generate a proactive assistance policy that synchronizes robotic actions with human activities, minimizing idle time and enhancing collaboration efficiency. We validate the framework through user studies involving 12 participants, showing significant improvements in interaction fluency, reduced waiting times, and positive user feedback compared to traditional methods.

Behavior-based Driver Identification is an emerging technology that recognizes drivers based on their unique driving behaviors, offering important applications such as vehicle theft prevention and personalized driving experiences. However, most studies fail to account for the real-world challenges of deploying Deep Learning models within vehicles. These challenges include operating under limited computational resources, adapting to new drivers, and changes in driving behavior over time. The objective of this study is to evaluate if Continual Learning (CL) is well-suited to address these challenges, as it enables models to retain previously learned knowledge while continually adapting with minimal computational overhead and resource requirements. We tested several CL techniques across three scenarios of increasing complexity based on a well-known dataset for the Driver Identification problem. This work provides an important step forward in scalable driver identification solutions, demonstrating that CL approaches, such as Dark Experience Replay (DER), can obtain strong performance with only an 11% reduction in accuracy compared to the static scenario. Furthermore, to enhance the performance, we propose two new methods, Smooth Experience Replay (SmooER) and Smooth Dark Experience Replay (SmooDER), that leverage the temporal continuity of driver identity over time to enhance classification accuracy. Our novel method, SmooDER, achieves optimal results with only a 2% accuracy reduction compared to the 11% of the DER approach. In conclusion, this study proves the feasibility of CL approaches to address the challenges of Driver Identification in dynamic environments, making them suitable for deployment on cloud infrastructure or directly within vehicles. • We investigate Driver Identification in a realistic setting, adapting to new drivers. • We propose three Continual Learning scenarios with progressive real-world alignment. • We propose SmooDER and SmooER, leveraging driver continuity to boost performance. • We validate the effectiveness of these techniques using the OCSLab dataset.

Deep Reinforcement Learning (DRL) has emerged as a powerful paradigm for learning complex policies directly from high-dimensional input spaces, enabling advances across a variety of domains. Modern DRL algorithms often rely on dual-network Q-learning architectures to approximate optimal policies to overcome overestimation bias. Recent research has introduced approaches leveraging multiple Q-functions to further mitigate overestimation effects and enhance policy reliability. However, there is a growing emphasis on deploying DRL in edge scenarios, where privacy concerns and stringent hardware constraints necessitate highly efficient algorithms. In such environments, the computational and memory efficiency of learning methods is of critical importance. In this context, we propose Edge Delayed Deep Deterministic Policy Gradient (EdgeD3), a novel reinforcement learning algorithm specifically designed for edge computing settings. EdgeD3 offers significant reductions in GPU time (by 25%) and computational and memory usage (by 30%), while consistently achieving or surpassing the performance of state-of-the-art algorithms across multiple benchmarks and in real-world tasks.

Learning the inverse dynamics of robots directly from data, adopting a black-box approach, is interesting for several real-world scenarios where limited knowledge about the system is available. In this article, we propose a black-box model based on Gaussian process (GP) regression for the identification of the inverse dynamics of robotic manipulators. The proposed model relies on a novel multidimensional kernel, calledLagrangian Inspired Polynomial(LIP) kernel. The LIP kernel is based on two main ideas. First, instead of directly modeling the inverse dynamics components, we model as GPs the kinetic and potential energy of the system. The GP prior on the inverse dynamics components is derived from those on the energies by applying the properties of GPs under linear operators. Second, as regards the energy prior definition, we prove a polynomial structure of the kinetic and potential energy, and we derive a polynomial kernel that encodes this property. As a consequence, the proposed model allows also to estimate the kinetic and potential energy without requiring any label on these quantities. Results on simulation and on two real robotic manipulators, namely a 7 DOF Franka Emika Panda, and a 6 DOF MELFA RV4FL, show that the proposed model outperforms state-of-the-art black-box estimators based both on Gaussian processes and neural networks in terms of accuracy, generality, and data efficiency. The experiments on the MELFA robot also demonstrate that our approach achieves performance comparable to fine-tuned model-based estimators, despite requiring less prior information. The code of the proposed model is publicly available.

In this paper, we investigate the use of ModelBased Reinforcement Learning (MBRL) for pick-and-throw tasks, which is a variation of the pick-and-place task. In pick-and-throw applications, objects are thrown instead of placed to increase time efficiency and enlarge the workspace reachable by the robot. The effectiveness of analytical methods for this task is limited by the complex system dynamics and the disturbances due to communication and control. Recently, the application of data-driven solutions, in particular Model-Free Reinforcement Learning (MFRL), has attracted attention. In this paper, we explore the use of a MBRL algorithm to solve the pick-and-throw task. Experiments carried out in simulation show the potentialities of our approach, in terms of accuracy and data efficiency, both with respect to analytical methods and a MFRL approach.

Hydroelectric is currently the most prominent among the sources of green energy, but, differently from the other sources, it has very strict requirements in terms of security that are taken into account with extremely robust constraints both at design and operations control times. In this paper, we evaluated the effectiveness of anomaly detection and explainability algorithms to supplement Decision Support System insights in Predictive Maintenance and Root Cause Analysis for hydroelectric power plants. The objective is to reduce operational costs and increase reliability in the plant, making hydroelectric technology more appealing to investors and promoting the transition to renewable energy. Specifically, the performance of several anomaly detection models was compared on real-world data with respect to the needs of the expert of the domain, that is the final user of the DSS, to work as an additional feature to speed up predictive maintenance. Additionally, the impact of SHapley Additive exPlanations values on helping the user understand the anomaly causes was investigated. Our findings are that the most performing algorithm was Auto-Encoder since it was able to find all recorded anomalies and even propose additional ones later confirmed by domain experts. The application of SHAP values was found to effectively guide the user toward the features related to the anomaly, although its application on streaming data was slow.

In this paper, we focus on the solution of online optimization problems that arise often in signal processing and machine learning, in which we have access to streaming sources of data. We discuss algorithms for online optimization based on the prediction-correction paradigm, both in the primal and dual space. In particular, we leverage the typical regularized least-squares structure appearing in many signal processing problems to propose a novel and tailored prediction strategy, which we call extrapolation-based. By using tools from operator theory, we then analyze the convergence of the proposed methods as applied both to primal and dual problems, deriving an explicit bound for the tracking error, that is, the distance from the time-varying optimal solution. We further discuss the empirical performance of the algorithm when applied to signal processing, machine learning, and robotics problems.

In this article, we present a model-based reinforcement learning (MBRL) algorithm namedMonte Carlo probabilistic inference for learning control(MC-PILCO). This algorithm relies on Gaussian processes (GPs) to model the system dynamics and on a Monte Carlo approach to estimate the policy gradient. This defines a framework in which we ablate the choice of the components, which are the selection of the cost function, the optimization of policies using dropout, and an improved data efficiency through the use of structured kernels in the GP models. The combination of the aforementioned aspects affects dramatically the performance of MC-PILCO. Numerical comparisons in a simulated cart–pole environment show that MC-PILCO exhibits better data efficiency and control performance w.r.t. state-of-the-art GP-based MBRL algorithms. Finally, we apply MC-PILCO to real systems, considering, in particular, systems with partially measurable states. We discuss the importance of modeling both the measurement system and the state estimators during policy optimization. The effectiveness of the proposed solutions has been tested in simulation and on two real systems, which are the Furuta pendulum and the ball-and-plate rig.

Robotics systems are becoming more and more autonomous and reconfigurable. In this context, the design of algorithms capable of deriving kinematics and dynamics models directly from data could be particularly useful. In this article, we present an algorithm that learns a forward kinematics model of a robot starting from a time series of visual observations. Our strategy can be applied to any robot with serial kinematics composed of revolute and prismatics joints. First, the algorithm identifies the robot kinematic structure, i.e., a high-level description of the robot geometry that defines the connections between the rigid-bodies composing the robot. Then, the algorithm derives the forward kinematics relying on a Gaussian process (GP) model. More precisely, the GP model is based on a polynomial kernel, defined exploiting the kinematic structure previously identified. The effectiveness of the proposed solution has been tested via extensive Monte Carlo simulations, as well as via experiments on a real UR10 robot.

We present an algorithm for minimizing the sum of a strongly convex time-varying function with a time-invariant, convex, and nonsmooth function. The proposed algorithm employs the prediction-correction scheme alongside the forward-backward envelope, and we are able to prove the convergence of the solutions to a neighborhood of the optimizer that depends on the sampling time. Numerical simulations for a time-varying regression problem with elastic net regularization highlight the effectiveness of the algorithm.

Wireless power transfer networks (WPTNs) are composed of dedicated energy transmitters (ETs) that charge energy receivers (ERs) via radio frequency waves. A safe-charging WPTN should keep electromagnetic radiation below predetermined limits meanwhile maximizing the transmitted power. In this paper, we consider this requirement as an optimization problem: the maximization of harvested power by ERs subject to the electro-magnetic safety constraints. In order to provide an approximated solution to this problem, we introduce a dual ascent-like distributed charging algorithm that enables ETs to work without global information and satisfy safety constraints asymptotically. We provide an in-depth theoretical analysis of our algorithm which is supported by numerical simulations.

A class of abstract aerial robotic systems is introduced, the laterally bounded force vehicles. in which most of the control authority is expressed along a principal thrust direction, while along the lateral directions a t smaller and possibly null) force may be exploited to achieve full-pose tracking. This class approximates platforms endowed with noncollinear rotors that can modify the orientation of the total thrust in a body frame. If made possible by the force constraints, the proposed SE(3)-based control strategy achieves the independent tracking of position-plus-orientation trajectories. The method, which is proven using a Lyapunov technique, deals seamlessly with both underactuated and fully actuated platforms, and guarantees at least the position tracking in the case of an unfeasible full-pose reference trajectory. Several experimental tests are presented that dearly show the approach practicability and the sharp improvement with respect to state of the art.

Harvesting-Based Wireless Sensor Devices are increasingly being deployed in today's sensor networks, due to their demonstrated advantages in terms of prolonged lifetime and autonomous operation. However, irreversible degradation mechanisms jeopardize battery lifetime, calling for intelligent management policies, which minimize the impact of these phenomena while guaranteeing a minimum Quality of Service (QoS). This paper explores a mathematical characterization of harvesting-based battery-powered sensor devices, focusing on the impact of the battery discharge policy on the irreversible degradation of the storage capacity. A general framework based on Markov chains which captures the battery degradation process is proposed. Based on such model, it is shown that a degradationaware policy significantly improves the lifetime of the sensor compared to "greedy" operation policies, while guaranteeing the minimum required QoS.

Energy Harvesting Wireless Sensor Devices are increasingly being considered for deployment in sensor networks, due to their demonstrated advantages of prolonged lifetime and autonomous operation. However, irreversible degradation mechanisms jeopardize battery lifetime, calling for intelligent management policies, which minimize the impact of these phenomena while guaranteeing a minimum Quality of Service (QoS). This paper explores a mathematical characterization of these devices, focusing on the interplay between the battery discharge policy and the irreversible degradation of the storage capacity. We propose a stochastic Markov chain framework, suitable for policy optimization, which captures the degradation status of the battery. We present a general result of Markov chains, which exploits the timescale separation between the communication time-slot of the device and the battery degradation process, and enables an efficient optimization. We show that this model fits well the behavior of real batteries for what concerns their storage capacity degradation over time. We demonstrate that a degradation-aware policy significantly improves the lifetime of the sensor compared to "greedy" policies, while guaranteeing the minimum required QoS. Finally, a simple heuristic policy, which never discharges the battery below a given threshold, is shown to achieve near-optimal performance in terms of battery lifetime.

Nowadays, many devices in wireless sensor networks are provided with energy harvesting capability to allow for their continuous operation over long periods of time. In principle, the energy level within each sensor should be managed optimally to ensure the best performance. Network engineers, however, often consider optimality under the idealized assumption of perfect knowledge about the State-of-Charge (SOC) of the device. This information is not always realistic or accurate. In our previous work [1], we showed that optimal policies for sensing, transmission, and battery usage should rather consider uncertainty on the SOC of the device. In this paper, we extend that investigation, therein performed in the idealized scenario of i.i.d. energy arrivals, by considering a correlated energy generation process. We show that the knowledge of the SOC and that of the energy generation process are useful in a complementary manner, that is they can be traded for each other. Moreover, the knowledge on the state of the energy generation process can obviate the need for acquiring accurate SOC information. This investigation paves the road for a new line of research in wireless sensor networks, allowing a tighter interaction between the designers of energy harvesting and battery storage mechanisms on the one hand, and the engineers of network operation and control policies on the other.

This paper studies prototypical strategies to sequentially aggregate independent decisions. We consider a collection of agents, each performing binary hypothesis testing and each obtaining a decision over time. We assume the agents are identical and receive independent information. Individual decisions are sequentially aggregated via a threshold-based rule. In other words, a collective decision is taken as soon as a specified number of agents report a concordant decision (simultaneous discordant decisions and no-decision outcomes are also handled). We obtain the following results. First, we characterize the probabilities of correct and wrong decisions as a function of time, group size, and decision threshold. The computational requirements of our approach are linear in the group size. Second, we consider the so-called fastest and majority rules, corresponding to specific decision thresholds. For these rules, we provide a comprehensive scalability analysis of both accuracy and decision time. In the limit of large group sizes, we show that the decision time for the fastest rule converges to the earliest possible individual time, and that the decision accuracy for the majority rule shows an exponential improvement over the individual accuracy. Additionally, via a theoretical and numerical analysis, we characterize various speed/accuracy tradeoffs. Finally, we relate our results to some recent observations reported in the cognitive information processing (CIP) literature.

We propose distributed algorithms to automatically deploy a team of mobile robots to partition and provide coverage of a nonconvex environment. To handle arbitrary nonconvex environments, we represent them as graphs. Our partitioning and coverage algorithm requires only short-range, unreliable pairwise “gossip” communication. The algorithm has two components: 1) a motion protocol to ensure that neighboring robots communicate at least sporadically and 2) a pairwise partitioning rule to update territory ownership when two robots communicate. By studying an appropriate dynamical system on the space of partitions of the graph vertices, we prove that territory ownership converges to a pairwise-optimal partition in finite time. This new equilibrium set represents improved performance over common Lloyd-type algorithms. Additionally, our algorithm is an "anytime algorithm'' that also scales well for large teams and can be run by on-board computers with limited resources. Finally, we report on large-scale simulations in complex environments and hardware experiments using the Player/Stage robot control system.

In this paper, we consider the problem of estimating the state of a dynamical system from distributed noisy measurements. Each agent constructs a local estimate based on its own measurements and on the estimates from its neighbors. Estimation is performed via a two stage strategy, the first being a Kalman-like measurement update which does not require communication, and the second being an estimate fusion using a consensus matrix. In particular we study the interaction between the consensus matrix, the number of messages exchanged per sampling time, and the Kalman gain for scalar systems. We prove that optimizing the consensus matrix for fastest convergence and using the centralized optimal gain is not necessarily the optimal strategy if the number of exchanged messages per sampling time is small. Moreover, we show that although the joint optimization of the consensus matrix and the Kalman gain is in general a non-convex problem, it is possible to compute them under some relevant scenarios. We also provide some numerical examples to clarify some of the analytical results and compare them with alternative estimation strategies.