Recent advances in Vision Language Models (VLMs) have enhanced their application in robotics, encompassing both high-level task planning and low-level action control. Despite their strong performance across various robotic tasks, even for zero-shot scenarios, most VLM applications remain open-loop, adhering to a plan-and-execute paradigm without mechanisms to assess task completion. To address this limitation, we propose GenCo, a Generate-Correct framework designed to automate a peg-in-hole task using a UR5e robot. This framework integrates an VLM-based motion generator and motion expert, working collaboratively to refine and correct actions during robotic task execution. Both VLM agents are fine-tuned using the pre-trained LLaVA, enhancing adaptability and scaling efficiently to diverse tasks. Our experiments demonstrate the adaptiveness of the framework, improving the success rate for the peg-in-hole task by 12.75% compared to a single VLM open-loop method. Notably, in unseen scenarios, the success rate for a triangular peg was increased by 15%, and for a random-shaped peg by 17%, underscoring the system's effectiveness in handling novel tasks. Adaptive testing under varied camera positions demonstrated robust performance, affirming reliability despite shifts in the visual input. The framework is also designed to be lightweight and efficient, facilitating broader adoption and practical deployment. Access to our code and model is provided here: https://github.com/Zhengxuez/generate_correct

This paper presents the details of a sub-system developed to address coordination between a serial manipulator robot (machining) and SwarmItFIX robot (fixturing) for a sheet metal drilling process. A heterogeneous multi-robot coordination methodology that has already been demonstrated to be successful in a milling process has been further enhanced here to make it suitable for a drilling process. For the convergence of joint angles in the trajectory planning of the serial manipulator robot, an optimisation-based approach is proposed. The velocity of the tool center point (TCP) is considered to be constant throughout, as it improves the quality of the machining. The SwarmItFIX robot abides by a revised five-step locomotion strategy to traverse between any two support locations. A new time plan that ensures multi-robot coordination has also been proposed in this work. The proposed method has been tested with three different drilling patterns, and the results show that the proposed method computes the trajectory of the serial manipulator, support locations of the SwarmItFIX and locomotion sequence of the base agent accurately.