Grasping and manipulation remain fundamental challenges in the effective deployment of robotic systems in real-world applications. In retail and supermarket scenarios, soft robotic grippers enable safe and efficient material handling. However, existing grasp planners are designed for rigid or suction-based grippers. Soft grasping is more challenging in terms of planning, estimation and sensing due to deflections in the gripper material on contact with the target. We present a system for soft robotic grasping using a custom gripper with a reconfigurable wrist that leverages reinforcement learning to augment existing vision-based techniques to adapt to the target object’s geometry. This system includes a hidden superquadrics module to guide the adaptation of the gripper’s palm configuration. We evaluate this system in a PyBullet simulation environment and compare it with a baseline synergy-based grasp strategy. Ongoing and future work involves transferring this planner to our physical robotic platform and evaluation in retail stores.

Managing shelves in retail stores includes re-stocking, rearrangement and replenishment of products. As these are some of the most labor-intensive activities, there has been widespread demand from retailers for automation in this domain. However, major challenges still remain in perception, navigation and manipulation while implementing an autonomous robotic system for this purpose. We present a system aimed at addressing some of these challenges through novel approaches. In terms of perception, we have developed a transformer-based local anomaly detection algorithm that can identify misplaced items without the need for a central database. Navigation of the omnidirectional mobile base is performed through stereo vision and LiDAR sensors. Finally, identifying grasping and manipulation as one of the key shortcomings of present robotic systems in this domain, we have developed a customized soft robotic gripper targeted at retail objects. It has compliant cable-driven fingers, and a palm configuration that can be adapted in real-time based on the target object's geometry. Coupled with a conventional two-fingered gripper in a dual-arm setup, this system is equipped to handle most objects encountered in a retail setting. We describe the underlying hardware and algorithms for each component of the system, evaluating their individual performance. We then evaluate the whole system in a mock retail setup, demonstrating promising results for autonomous management of shelves.

The swiftly expanding retail sector is increasingly adopting autonomous mobile robots empowered by artificial intelligence and machine learning algorithms to gain an edge in the competitive market. However, these autonomous robots encounter challenges in adapting to the dynamic nature of retail products, often struggling to operate autonomously in novel situations. In this study, we introduce an omni-directional dual-arm mobile robot specifically tailored for use in retail environments. Additionally, we propose a tele-operation method that enables shared control between the robot and a human operator. This approach utilizes a Virtual Reality (VR) motion capture system to capture the operator's commands, which are then transmitted to the robot located remotely in a retail setting. Furthermore, the robot is equipped with heterogeneous grippers on both manipulators, facilitating the handling of a wide range of items. We validate the efficacy of the proposed system through testing in a mockup of retail environment, demonstrating its ability to manipulate various commonly encountered retail items using both single and dual-arm coordinated manipulation techniques.

Tracking of inventory and rearrangement of mis-placed items are some of the most labor-intensive tasks in a retail environment. While there have been attempts at using vision-based techniques for these tasks, they mostly use planogram compliance for detection of any anomalies, a technique that has been found lacking in robustness and scalability. Moreover, existing systems rely on human intervention to perform corrective actions after detection. In this paper, we present Co-AD, a Concept-based Anomaly Detection approach using a Vision Transformer (ViT) that is able to flag misplaced objects without using a prior knowledge base such as a planogram. It uses an auto-encoder architecture followed by outlier detection in the latent space. Co-AD has a peak success rate of 89.90% on anomaly detection image sets of retail objects drawn from the RP2K dataset, compared to 80.81% on the best-performing baseline of a standard ViT auto-encoder. To demonstrate its utility, we describe a robotic mobile manipulation pipeline to autonomously correct the anomalies flagged by Co-AD. This work is ultimately aimed towards developing autonomous mobile robot solutions that reduce the need for human intervention in retail store management.