High-resolution visuotactile sensors provide detailed contact information that is promising to infer the physical properties of objects in contact. This paper introduces a novel technique for high-resolution stiffness estimation of heterogeneous deformable objects using the Punyo bubble sensor. We developed an observation model for dense contact forces to estimate object stiffness using a visuotactile sensor and a dense force estimator. Additionally, we propose a neural Volumetric Stiffness Field (VSF) formulation that represents stiffness as a continuous function, which allows dynamic point sampling at visuotactile sensor observation resolution. The neural VSF significantly reduces artifacts commonly associated with traditional point-based methods, particularly in stiff inclusion estimation and heterogeneous stiffness estimation. We further apply our method in a blind localization task, where objects within opaque bags are accurately modeled and localized, demonstrating the superior performance of neural VSF compared to existing techniques. Project page: https://hjh371.github.io/Neural-VSF/.

Fruit monitoring plays an important role in crop management, and rising global fruit consumption combined with labor shortages necessitates automated monitoring with robots. However, occlusions from plant foliage often hinder accurate shape and pose estimation. Therefore, we propose an active fruit shape and pose estimation method that physically manipulates occluding leaves to reveal hidden fruits. This paper introduces a framework that plans robot actions to maximize visibility and minimize leaf damage. We developed a novel scene-consistent shape completion technique to improve fruit estimation under heavy occlusion and utilize a perception-driven deformation graph model to predict leaf deformation during planning. Experiments on artificial and real sweet pepper plants demonstrate that our method enables robots to safely move leaves aside, exposing fruits for accurate shape and pose estimation, outperforming baseline methods. Project page: https://shaoxiongyao.github.io/lmap-ssc/.

Soft-bubble tactile sensors have the potential to capture dense contact and force information across a large contact surface. However, it is difficult to extract contact forces directly from observing the bubble surface because local contacts change the global surface shape significantly due to membrane mechanics and air pressure. This paper presents a model-based method of reconstructing dense contact forces from the bubble sensor’s internal RGBD camera and air pressure sensor. We present a finite element model of the force response of the bubble sensor that uses a linear plane stress approximation that only requires calibrating 3 variables. Our method is shown to reconstruct normal and shear forces significantly more accurately than the state-of-the-art, with comparable accuracy for detecting the contact patch, and with very little calibration data.

This paper introduces a method for learning the force response of heterogeneous, deformable objects directly from robot sensor data without prior knowledge. The method estimates an object's force response given robot force or torque measurements using a novel volumetric stiffness field representation and point-based contact simulator. The stiffness of each point colliding with the robot is estimated independently and is updated upon each observed measurement using a projected diagonal Kalman filter. Experiments show that this method can update a stiffness field over 105points at 23 Hz or higher, and is more accurate than learning-based methods in predicting torque response while touching artificial plants. The method can also be augmented with visual information to help extrapolate stiffness fields to distant parts of the touched object using only a small number of touches.

RFID-based passive positioning is essential for many Internet-of-Things applications. However, existing RFID-based passive positioning systems either have low precision or require multiple antenna arrays, which are bulky and expensive. In this paper, a 3D passive positioning scheme is developed based on RFID tag arrays. One of the tags can harvest the energy of the surrounding electromagnetic waves to power the embedded orientation sensors (an accelerometer and a magnetometer) and then send the orientation information of an object to an RFID reader. When an object is equipped with such RFID tag arrays, its direction can be estimated by exploring the phases of the signals reflected by the tags and the orientation of the object. Furthermore, based on the triangulation principle, the 3D location of an object can be determined by using multiple RFID tags on the object and two antennas at the RFID reader. Thus, compared with existing schemes, the required number of RFID antennas for 3D passive positioning is greatly reduced. The proposed scheme is implemented based on a COTS RFID platform. The experimental results show that the 90-percentile accuracy of the proposed system is within 9 centimeters.