Pedestrian detection is one of the most explored topics in computer vision and robotics. The use of deep learning methods allowed the development of new and highly competitive algorithms. Deep Reinforcement Learning has proved to be within the state-of-the-art in terms of both detection in perspective cameras and robotics applications. However, for detection in omnidirectional cameras, the literature is still scarce, mostly because of their high levels of distortion. This paper presents a novel and efficient technique for robust pedestrian detection in omnidirectional images. The proposed method uses deep Reinforcement Learning that takes advantage of the distortion in the image. By considering the 3D bounding boxes and their distorted projections into the image, our method is able to provide the pedestrian’s position in the world, in contrast to the image positions provided by most state-of-the-art methods for perspective cameras. Our method avoids the need of pre-processing steps to remove the distortion, which is computationally expensive. Beyond the novel solution, our method compares favorably with the state-of-the-art methodologies that do not consider the underlying distortion for the detection task.

In this paper we present a framework for the application of augmented reality to a mobile robot, using non-central camera systems. Considering a virtual object in the world with known local 3D coordinates, the goal is to project this object into the image of a non-central catadioptric imaging device. We propose a solution to this problem which allows us to project textured objects to the image in real-time (up to 20 fps): projection of 3D segments to the image; occlusions; and illumination. In addition, since we are considering that the imaging device is on a mobile robot, one needs to take into account the real-time localization of the robot. To the best of our knowledge this is the first time that this problem is addressed (all state-of-the-art methods are derived for central camera systems). To evaluate the proposed framework we test the solution using a mobile robot and a non-central catadioptric camera (using a spherical mirror).