First prototypes of standard plenoptic cameras (SPCs) were based on arrays of pinhole cameras. Despite the array nature, viewpoint pinhole arrays are not intrinsically provided by current SPC calibration tools. In this work, we start by detailing the mapping between the SPC model and a camera array of viewpoints. Then, the mapping is used to propose a calibration procedure for the SPC based on a grid of corners. Calibration involves two steps, first a linear solution and then a nonlinear optimization minimizing the ray re-projection error. The proposed calibration methodology compares favourably with state of the art calibrations and the linear solution proposed for the initial stage of the calibration outperforms the state of the art.

Humanoid robots are complex sensorimotor systems where the existence of internal models are of utmost importance both for control purposes and for predicting the changes in the world arising from the system's own actions. This so-called expected perception relies on the existence of accurate internal models of the robot's sensorimotor chains.

The maintenance of a stable and coherent representation of the surrounding environment is an essential capability in cognitive robotic systems. Most systems employ some form of 3D perception to create internal representations of space (maps) to support tasks such as navigation, manipulation and interaction. The creation and update of such representations may represent a significant effort in the overall computation performed by the robot. In this paper we propose an architecture based on the concept of Expected Perception that allows lightweight map updates whenever the course of action happens according to the robot's expectations. It is only when the robot's predictions and the real world outcomes differ, that corrections must be done at its full extent. We performed experiments and show results in a real robotic platform with stereo (3D) perception where map corrections are proposed by simple image level (2D) comparisons.

This paper introduces a testbed for sensor and robot network systems, currently composed of 10 cameras and 5 mobile wheeled robots equipped with several sensors for self-localization, obstacle avoidance and vision cameras, and wireless communications. The testbed includes a service-oriented middleware to enable fast prototyping and implementation of algorithms previously tested in simulation, as well as to simplify integration of subsystems developed by different partners. We survey an integrated approach to human-robot interaction that has been developed supported by the testbed under an European research project. The application integrates innovative methods and algorithms for people tracking and waving detection, cooperative perception among static and mobile cameras to improve people tracking accuracy, as well as decision-theoretical approaches to sensor selection and task allocation within the sensor network.

Outdoor camera networks are becoming ubiquitous in critical urban areas of large cities around the world. Although current applications of camera networks are mostly limited to video surveillance, recent research projects are exploiting advances on outdoor robotics technology to develop systems that put together networks of cameras and mobile robots in people assisting tasks. Such systems require the creation of robot navigation systems in urban areas with a precise calibration of the distributed camera network. Despite camera calibration has been an extensively studied topic, the calibration (intrinsic and extrinsic) of large outdoor camera networks with no overlapping view fields, and likely to suffer frequent recalibration, poses novel challenges in the development of practical methods for user-assisted calibration that minimize intervention times and maximize precision. In this paper we propose the utilization of Laser Range Finder (LRF) data covering the area of the camera network to support the calibration process and develop a semi-automated methodology allowing quick and precise calibration of large camera networks. The proposed methods have been tested in a real urban environment and have been applied to create direct mappings (homographies) between image coordinates and world points in the ground plane (walking areas) to support person and robot detection and localization algorithms.

This paper presents a method for 3D model-based tracking of colored objects using a sampling methodology. The problem is formulated in a Monte Carlo filtering approach, whereby the state of an object is represented by a set of hypotheses. The main originality of this work is an observation model consisting in the comparison of the color information in some sampling points around the target’s hypothetical edges. On the contrary to existing approaches the method does not need to explicitly compute edges in the video stream, thus dealing well with optical or motion blur. The method does not require the projection of the full 3D object on the image, but just of some selected points around the target’s boundaries. This allows a flexible and modular architecture illustrated by experiments performed with different objects (balls and boxes), camera models (perspective, catadioptric, dioptric) and tracking methodologies (particle and Kalman filtering). 1

We consider the problem of estimating the relative orientation of a number of individual photocells-or pixels-that hold fixed relative positions. The photocells measure the intensity of on a pencil of lines. We assume that the light-field thus sampled is changing, e.g. as the result of motion of the sensors and use the obtained measurements to estimate the orientations of the photocells. We explore an information-theoretic and geometric approach: based on real-world data, we build anon-parametric functional relation linking the information distance between the data streams of two photocells, and the angular separation between the photocells. Then, given data streams produced by arrays of pixels in similar conditions, we use the functional relation to estimate the angles between pixels. Finally, we embed the estimated angles in the unit 3D sphere to obtain the estimated layout of the array.

We present a model-based 3D tracking system, using wide angle perspective catadioptric sensors. These sensors acquire 360deg views of the environment and the projection from 3D world points to the image plane is approximated by a perspective model. This is a major advantage in structured environments because straight lines on specific surfaces are not deformed by the sensor, allowing the application of standard computer vision algorithms. Objects off the surface are distorted according to a complex projection model, but can be approximated by a simple wide angle perspective mapping. This is exploited here to develop a robust tracking system for autonomous robots using a 3D shape and color-based object model. The use of particle filters allows tracking to be done with 3D realistic motion models and tackling object occlusion, overlap and ambiguities. We show that the use of the perspective model is advantageous over more standard catadioptric projection models, since it renders a very good approximation to the true model, being simpler and more efficient to use, in particular with 3D particle filtering methods.

This paper presents a method to cooperatively localize pairs of robots fusing bearing-only information provided by cameras and the motion of the vehicles. The algorithm uses the robots as landmarks to estimate their relative location. Bearings are the simplest measurements directly obtained from the cameras, as opposed to measuring depths which would require knowledge or reconstruction of the world structure. We present the general recursive Bayes estimator and three different implementations based on an extended Kalman filter, a particle filter and a combination of both techniques. We have compared the performance of the different implementations using real data acquired with two platforms equipped with omnidirectional cameras and simulated data.

Proposes a method for the visual-based navigation of a mobile robot in indoor environments, using a single omnidirectional (catadioptric) camera. The geometry of the catadioptric sensor and the method used to obtain a bird's eye (orthographic) view of the ground plane are presented. This representation significantly simplifies the solution to navigation problems, by eliminating any perspective effects. The nature of each navigation task is taken into account when designing the required navigation skills and environmental representations. We propose two main navigation modalities: topological navigation and visual path following. Topological navigation is used for traveling long distances and does not require knowledge of the exact position of the robot but rather, a qualitative position on the topological map. The navigation process combines appearance based methods and visual servoing upon some environmental features. Visual path following is required for local, very precise navigation, e.g., door traversal, docking. The robot is controlled to follow a prespecified path accurately, by tracking visual landmarks in bird's eye views of the ground plane. By clearly separating the nature of these navigation tasks, a simple and yet powerful navigation system is obtained.