The wheelchair-mounted robotic arm (WMRA) is a mobile manipulator that consists of a 7-DoF robotic arm and a 2-DoF power wheelchair platform. Previous works combined mobility and manipulation control using weighted optimization for dual-trajectory tracking [7]. In this work, we present an image-based visual servoing (IBVS) approach with scale-invariant feature transform (SIFT) using an eye-in-hand monocular camera for combined control of mobility and manipulation for the 9-DoF WMRA system to execute activities of daily living (ADL) autonomously. We also present results of the physical implementation with a simple “Go to and Pick Up” task and the “Go to and Open the Door” task previously published in simulation, using IBVS to aid the task performance.

Wheelchair-bound persons with upper limb motion limitations can utilize a wheelchair-mounted robotic arm (WMRA) to perform activities of daily living (ADL) tasks. In this paper, an optimized control of our 9-DoF system, consisting of a 7-DoF robotic arm and a 2-DoF power wheelchair, is achieved. For effective ADL task execution, positioning the end-effector with proper wheelchair orientation was optimized as part of the control algorithm. Separate wheelchair and end-effector trajectories were simultaneously followed to execute a “Go To and Open the Door” task. The control methodology, implementation and test results in simulation are presented in this paper.