Many human activities are performed in groups—making decisions in workplace meetings, cooperating on a sports team, or meeting with friends for dinner. All these activities involve complex conditions and interaction processes that influence their outcomes in terms of performance, personal goals, and group objectives. As robots are increasingly being positioned within groups, improving these outcomes has emerged as an important application area in social robotics, particularly through robotic facilitation. Robot facilitators aim to elicit positive changes by deliberately influencing group processes. While research in this field has demonstrated that robots can effectively influence interpersonal dynamics, there remains a notable gap in consolidating these insights into a coherent understanding that can guide the design and development of better facilitators. We present a scoping review of literature targeting changes in interactions between multiple humans that are driven by intentional actions from robotic agents. To identify key considerations for the design of robot facilitators, we take inspiration from human group research theories to organize existing approaches. Our review includes 108 publications that meet our inclusion criteria, yielding 85 distinct application targets for group facilitation using robots. Based on the identified instances, we extract categories of possible application targets and a set of design concepts that can guide future work on robotic group facilitators.

To achieve dexterity comparable to that of humans, robots must intelligently process tactile sensor data. Taxel-based tactile signals often have low spatial-resolution, with non-standardized representations. In this paper, we propose a novel framework, HyperTaxel, for learning a geometrically-informed representation of taxel-based tactile signals to address challenges associated with their spatial resolution. We use this representation and a contrastive learning objective to encode and map sparse low-resolution taxel signals to high-resolution contact surfaces. To address the uncertainty inherent in these signals, we leverage joint probability distributions across multiple simultaneous contacts to improve taxel hyper-resolution. We evaluate our representation by comparing it with two baselines and present results that suggest our representation outperforms the baselines. Furthermore, we present qualitative results that demonstrate the learned representation captures the geometric features of the contact surface, such as flatness, curvature, and edges, and generalizes across different objects and sensor configurations. Moreover, we present results that suggest our representation improves the performance of various downstream tasks, such as surface classification, 6D in-hand pose estimation, and sim-to-real transfer.

This paper introduces CORAE, a novel web-based open-source tool for COntinuous Retrospective Affect Evaluation, designed to capture continuous affect data about interpersonal perceptions in dyadic interactions. Grounded in behavioral ecology perspectives of emotion, this approach replaces valence as the relevant rating dimension with approach and withdrawal, reflecting the degree to which behavior is perceived as increasing or decreasing social distance. We conducted a study to experimentally validate the efficacy of our platform with 24 participants. The tool’s effectiveness was tested in the context of dyadic negotiation, revealing insights about how interpersonal dynamics evolve over time. We find that the continuous affect rating method is consistent with individuals’ perception of the overall interaction. This paper contributes to the growing body of research on affective computing and offers a valuable tool for researchers interested in investigating the temporal dynamics of affect and emotion in social interactions.

Robotic manipulation, in particular in-hand object manipulation, often requires an accurate estimate of the object's 6D pose. To improve the accuracy of the estimated pose, state-of-the-art approaches in 6D object pose estimation use observational data from one or more modalities, e.g., RGB images, depth, and tactile readings. However, existing approaches make limited use of the underlying geometric structure of the object captured by these modalities, thereby, increasing their reliance on visual features. This results in poor performance when presented with objects that lack such visual features or when visual features are simply occluded. Furthermore, current approaches do not take advantage of the proprioceptive information embedded in the position of the fingers. To address these limitations, in this paper: (1) we introduce a hierarchical graph neural network architecture for combining multimodal (vision and touch) data that allows for a geometrically informed 6D object pose estimation, (2) we introduce a hierarchical message passing operation that flows the information within and across modalities to learn a graph-based object representation, and (3) we introduce a method that accounts for the proprioceptive information for in-hand object representation. We evaluate our model on a diverse subset of objects from the YCB Object and Model Set, and show that our method substantially outperforms existing state-of-the-art work in accuracy and robustness to occlusion. We also deploy our proposed framework on a real robot and qualitatively demonstrate successful transfer to real settings.

Robotic fabric manipulation is challenging due to the infinite dimensional configuration space, self-occlusion, and complex dynamics of fabrics. There has been significant prior work on learning policies for specific fabric manipulation tasks, but comparatively less focus on algorithms which can perform many different tasks. We take a step towards this goal by learning point-pair correspondences across different fabric configurations in simulation. Then, given a single demonstration of a new task from an initial fabric configuration, these correspondences can be used to compute geometrically equivalent actions in a new fabric configuration. This makes it possible to define policies to robustly imitate a broad set of multi-step fabric smoothing and folding tasks. The resulting policies achieve 80.3% average task success rate across 10 fabric manipulation tasks on two different physical robotic systems. Results also suggest robustness to fabrics of various colors, sizes, and shapes. See https://tinyurl.com/fabric-descriptors for supplementary material and videos.

Tactile sensing is inherently contact based. To use tactile data, robots need to make contact with the surface of an object. This is inefficient in applications where an agent needs to make a decision between multiple alternatives that depend the physical properties of the contact location. We propose a method to get tactile data in a non-invasive manner. The proposed method estimates the output of a tactile sensor from the depth data of the surface of the object based on past experiences. An experience dataset is built by allowing the robot to interact with various objects, collecting tactile data and the corresponding object surface depth data. We use the experience dataset to train a neural network to estimate the tactile output from depth data alone. We use GelSight tactile sensors, an image-based sensor, to generate images that capture detailed surface features at the contact location. We train a network with a dataset containing 578 tactile-image to depth- map correspondences. Given a depth-map of the surface of an object, the network outputs an estimate of the response of the tactile sensor, should it make a contact with the object. We evaluate the method with structural similarity index matrix (SSIM), a similarity metric between two images commonly used in image processing community. We present experimental results that show the proposed method outperforms a baseline that uses random images with statistical significance getting an SSIM score of 0.84 ± 0.0056 and 0.80 ± 0.0036, respectively.

Sequential pulling policies to flatten and smooth fabrics have applications from surgery to manufacturing to home tasks such as bed making and folding clothes. Due to the complexity of fabric states and dynamics, we apply deep imitation learning to learn policies that, given color (RGB), depth (D), or combined color-depth (RGBD) images of a rectangular fabric sample, estimate pick points and pull vectors to spread the fabric to maximize coverage. To generate data, we develop a fabric simulator and an algorithmic supervisor that has access to complete state information. We train policies in simulation using domain randomization and dataset aggregation (DAgger) on three tiers of difficulty in the initial randomized configuration. We present results comparing five baseline policies to learned policies and report systematic comparisons of RGB vs D vs RGBD images as inputs. In simulation, learned policies achieve comparable or superior performance to analytic baselines. In 180 physical experiments with the da Vinci Research Kit (dVRK) surgical robot, RGBD policies trained in simulation attain coverage of 83% to 95% depending on difficulty tier, suggesting that effective fabric smoothing policies can be learned from an algorithmic supervisor and that depth sensing is a valuable addition to color alone. Supplementary material is available at https://sites.google.com/view/fabric-smoothing.

We propose a method to compress the enormous amount of data originating from tactile sensors is presented that explicitly exploits the inherent sparseness over space and time, sending tactile “events” only when a contact is detected. The resulting modular architecture is based on FPGA modules that acquire data samples from off-the-shelf tactile sensors based on capacitive transducers and generate and transmit an event-driven readout. This architecture has been specifically implemented for integration on robots with a large number of tactile sensors, to reduce communication bandwidth, power and processing requirements. An asynchronous serial address-event representation protocol further optimises effective data transmission rate (efficiency of 94.1%) and latency (340 ns) with respect to more common transmission protocols (e.g., Ethernet, CAN). We propose two complementary algorithms for the translation of raw-data into events, optimising data rate and bandwidth, or exploiting the asynchronous nature of the event-driven encoding and the temporal information within the sensory signal. Data reduction capability can reach up to 20 % of the correspondent clock-based encoding, with limited information loss due to the compression.

Autonomous manipulation of objects requires reliable information on robot-object contact state. Underwater environments can adversely affect sensing modalities such as vision, making them unreliable. In this paper we investigate underwater robot-object contact perception between an autonomous underwater vehicle and a T-bar valve using a force/torque sensor and the robot's proprioceptive information. We present an approach in which machine learning is used to learn a classifier for different contact states, namely, a contact aligned with the central axis of the valve, an edge contact and no contact. To distinguish between different contact states, the robot performs an exploratory behavior that produces distinct patterns in the force/torque sensor. The sensor output forms a multidimensional time-series. A probabilistic clustering algorithm is used to analyze the time-series. The algorithm dissects the multidimensional time-series into clusters, producing a one-dimensional sequence of symbols. The symbols are used to train a hidden Markov model, which is subsequently used to predict novel contact conditions. We show that the learned classifier can successfully distinguish the three contact states with an accuracy of 72% ± 12 %.

Tactile sensing is of fundamental importance for object manipulation and perception. Several sensors for hands have been proposed in the literature, however, only a few of them can be fully integrated with robotic hands. Typical problems preventing integration include the need for deformable sensors that can be deployed on curved surfaces, and wiring complexity. In this paper we describe a fingertip for the hands of the iCub robot, each fingertip consists of 12 sensors. Our approach builds on previous work on the iCub tactile system. The sensing elements of the fingertip are capacitive sensors made from a flexible PCB, and a multi-layer fabric that includes the dielectric material and the conductive layer. The novelty the proposed sensor lies in incorporating the multi-layer fabric technology into a small fingertip sensor that can be attached to the hands of a humanoid robot. The new sensors are more robust. The manufacturing is easier and relies on industrial techniques for the fabrication of the components, which results in higher repeatability. We performed experimental characterization of the sensor. We show that the sensor is able to detect forces as low as 0.05 N with no cross-talk between the taxels. We identified some hysteresis in the response of the sensor which must be taken into account if the robot exerts large forces for a long period of time. The taxels have spatially overlapping receptive fields, this has been demonstrated to be a useful property that allows hyperacuity.

This paper investigates application of machine learning to the problem of contact perception between a robot's gripper and an object. The input data comprises a multidimensional time-series produced by a force/torque sensor at the robot's wrist, the robot's proprioceptive information, namely, the position of the end-effector, as well as the robot's control command. These data are used to train a hidden Markov model (HMM) classifier. The output of the classifier is a prediction of the contact state, which includes no contact, a contact aligned with the central axis of the valve, and an edge contact. To distinguish between contact states, the robot performs exploratory behaviors that produce distinct patterns in the time-series data. The patterns are discovered by first analyzing the data using a probabilistic clustering algorithm that transforms the multidimensional data into a one-dimensional sequence of symbols. The symbols produced by the clustering algorithm are used to train the HMM classifier. We examined two exploratory behaviors: a rotation around the x-axis, and a rotation around the y-axis of the gripper. We show that using these two exploratory behaviors we can successfully predict a contact state with an accuracy of 88 ± 5 % and 81 ± 10 %, respectively.

We present experiments on the application of machine learning to predicting slip. The sensing information is provided by a force/torque sensor and an artificial finger, which has randomly distributed strain gauges and polyvinylidene fluoride (PVDF) films embedded in silicone resulting in multidimensional time-series data on the finger-object contact. An incipient slip is detected by studying temporal patterns in the data. The data is analysed using probabilistic clustering that transforms the data into a sequence of symbols, which is used to train a hidden Markov model (HMM) classifier. Experimental results show that the classifier can predict a slip, at least 100ms before a slip takes place, with an accuracy of 96% on the validation set.

In this paper, we present an application of machine learning to distinguish between different materials based on their surface texture. Such a system can be used for the estimation of surface friction during manipulation tasks; quality assurance in the textile, cosmetics, and harvesting industries; and other applications requiring tactile sensing. Several machine learning algorithms, such as naive Bayes, decision trees, and naive Bayes trees, have been trained to distinguish textures sensed by a biologically inspired artificial finger. The finger has randomly distributed strain gauges and polyvinylidene fluoride (PVDF) films embedded in silicone. Different textures induce different intensities of vibrations in the silicone. Consequently, textures can be distinguished by the presence of different frequencies in the signal. The data from the finger are preprocessed, and the Fourier coefficients of the sensor outputs are used to train classifiers. We show that the classifiers generalize well for unseen datasets with performance exceeding previously reported algorithms. Our classifiers can distinguish between different materials, such as carpet, flooring vinyls, tiles, sponge, wood, and polyvinyl-chloride (PVC) woven mesh with an accuracy of on unseen test data.

In this paper we describe an application of machine learning to distinguish between seven different materials, based on their surface texture. Applications of such a system includes quality assurance and estimating surface friction during manipulation tasks. A naive Bayes classifier is used to distinguish textures sensed by a bio-inspired artificial finger. The finger has randomly distributed strain gauges and Polyvinylidene Fluoride (PVDF) films embedded in silicone. Different textures induce different intensity of vibrations in the silicone. Textures can be distinguished by the presence of different frequencies in the signal. The data from the finger is pre-processed and the Fourier coefficients of the sensor outputs are used to learn a classifier for different textures. The performance of the classifier is evaluated against a naive time domain based learner. Preliminary results show that our classifier performs better.