Large-scale evaluation of state-of-the-art visual SLAM has shown that its tracking performance degrades considerably if the camera view is not adjusted to avoid the low-texture areas. Deep reinforcement learning based approaches have been proposed to improve the robustness of visual tracking in such unsupervised settings. Our extensive analysis reveals the fundamental limitations of reinforcement learning based active view planning, especially in transition scenarios (entering/exiting the room, texture-less walls, and lobbies). In challenging transition scenarios, the agent generally remains unable to cross the transition during training, limiting its ability to learn the maneuver. We propose human-supervised RL training (imitation learning) and achieve significantly improved performance after$\sim$50 hours of supervised training. To reduce longer human supervision requirements, we also explore fine-tuning our network with an online learning policy. Here, we use limited human-supervised training ($\sim$20 hours), and fine-tune the network with unsupervised training ($\sim$45 hours), obtaining encouraging results. We also release our multi-model, human supervised training dataset. The dataset contains challenging and diverse transition scenarios and can aid the development of imitation learning policies for consistent visual tracking. We also release our implementation11https://tinyurl.com/478maz6u.

Large scale active exploration has recently revealed limitations of visual SLAM’s tracking ability. Active view planning methods based on reinforcement learning have been proposed to improve visual tracking robustness.In this work, we expose the limitations of deep reinforcement learning-based visual SLAM over longer routes. We demonstrate that additional modalities (depth, scene layout) offer little improvement. Furthermore, reward shaping is not the main reason behind the shortsightedness of the state-of-the-art visual SLAM tracker. We propose a novel video vision transformer-based architecture that improves the farsightedness of the visual tracker, which results in the completion of longer routes with efficient paths.Out of 60 challenging routes, our approach manages to complete 56 routes, which is a three-fold improvement over the state-of-the-art active view mapping (DI-SLAM) baseline. Interestingly, ORB-SLAM3 was unable to complete a single route without tracking failure. Our code is available at https://tinyurl.com/w935spuz.

This paper presents a vision tracking system to achieve high recognition performance under dynamic circumstances, using a fuzzy logic controller. The main concept of the proposed system is based on the vestibulo-ocular reflex (VOR) and the opto-kinetic reflex (OKR) of the human eye. To realize the VOR concept, MEMS inertial sensors and encoders are used for robot motion detection. This concept turns the camera towards a selected target, counteracting the robot motion. Based on the OKR concept, the targeting errors are periodically compensated, using vision information. The fuzzy logic controller uses sensor data fusion to detect slip or collision occurrences. To calculate a heading angle of the camera accurately, the output of the fuzzy logic controller and the vision information from the camera are combined, using an extended Kalman filter. The proposed vision tracking system is implemented in a mobile robot and evaluated experimentally. The experimental results are obtained as the tracking and the recognition success rate using a mobile robot. The developed system achieved the excellent tracking and recognition performance during slip or collision occurrences under dynamic circumstances.

This paper introduces a high performance vision tracking system for mobile robot using sensor data fusion. For mobile robots, it is difficult to collect continuous vision information due to robot's motion. To solve this problem, the proposed vision tracking system estimates the robot's position relative to a target and rotates the camera towards the target. This concept is derived from the human eye reflex mechanism, known as the Vestibulo-Ocular Reflex (VOR), for compensating the head motion. This concept for tracking the target results in much higher performance levels, when compared with the conventional method that rotates the camera using only vision information. The proposed system do not require heavy computing loads to process image data and can track the target continuously even during vision occlusion. The robot motion information is estimated using data from accelerometer, gyroscope, and encoders. This multi-sensor data fusion is achieved using Kalman filter. The proposed vision tracking system is implemented on a two-wheeled robot. The experimental results show that the proposed system achieves excellent tracking and recognition performance in various motion scenarios, including scenarios where camera is temporarily blocked from the target.