This paper presents a novel data-driven crowd simulation method that can mimic the observed traffic of pedestrians in a given environment. Given a set of observed trajectories, we use a recent form of neural networks, Generative Adversarial Networks (GANs), to learn the properties of this set and generate new trajectories with similar properties. We define a way for simulated pedestrians (agents) to follow such a trajectory while handling local collision avoidance. As such, the system can generate a crowd that behaves similarly to observations, while still enabling real-time interactions between agents. Via experiments with real-world data, we show that our simulated trajectories preserve the statistical properties of their input. Our method simulates crowds in real time that resemble existing crowds, while also allowing insertion of extra agents, combination with other simulation methods, and user interaction.

We present the Regression-based Linear Quadratic Regulator (R-LQR), a new approach for determining locally-optimal control feedback policies for robots with non-linear dynamics and non-quadratic cost functions. Our proposal uses a free-derivative algorithm based on local quadratic regressions to obtain the robot motion policy. In addition, our methodology allows to define a notion of scale that translates into the definition of neighborhoods of valid policy and into the exploration of larger areas of the search space to find the optimal policies. The results show that our formulation allows to reach policies with lower costs than existing algorithms and to avoid problems when the behavior of the cost function makes the optimization difficult.

We propose a novel real-time algorithm for estimating the local scale correction of a monocular SLAM system, to obtain a correctly scaled version of the 3D map and of the camera trajectory. Within a Bayesian framework, it integrates observations from a deep-learning based generic object detector and landmarks from the map whose projection lie inside a detection region, to produce scale correction estimates from single frames. For each observation, a prior distribution on the height of the detected object class is used to define the observation's likelihood. Due to the scale drift inherent to monocular SLAM systems, we also incorporate a rough model on the dynamics of scale drift. Quantitative evaluations are presented on the KITTI dataset, and compared with different approaches. The results show a superior performance of our proposal in terms of relative translational error when compared to other monocular systems based on object detection.

We propose a methodology for learning and using a multiple-goal probabilistic motion model within a particle filter-based target tracking on video streams. In a set of training video sequences, we first extract the locations (coined as “goals”) where the pedestrians either leave the scene or often change directions. Then, we learn one motion prior model per detected goal. Each of these models is learned statistically based on the local motion observed by the camera during the training phase. Given that the initial, empirical distribution may be incomplete or noisy, we regularize it in a second phase. These priors are then used in an Interactive Multiple Model (IMM) scheme for target tracking and goal estimation. We demonstrate the relevance of this methodology with tracking experiments and comparisons done on standard datasets.

Even though pedestrian motion may look chaotic in most of the cases, recent studies have shown that this motion is mainly ruled by environment and social aspects. In this paper, we propose an interacting multiple model pedestrian tracking framework that incorporates these semantic considerations as a prior knowledge about intentions and interactions between targets. We consider 4 cases of motion for pedestrians: going straight, finding one's way, walking around and standing still. Those models are competing within an Interacting Multiple Model Particle Filter strategy. Targets interactions are handled with social forces, included as potential functions in the weighting process of the Particle Filter (PF). We use different social force models in each motion model to handle high level behaviors (collision avoidance, flocking...). We evaluate our algorithm on challenging datasets and demonstrate that such semantic information improves the tracker performance.

We present a novel approach to introduce visual information in the walking pattern generator for humanoid robots in a more direct way than the current existing methods. We make use of a model predictive control (MPC) visual servoing strategy, which is combined to the walking motion generator. We define two schemes based on that principle: a position-based and an image-based scheme, with a Quadratic Program (QP) formulation in both cases. Finally, we present some simulation results validating our approach.

Multiple targets tracking is a challenging problem due to occlusions or identity switching. Although the use of prior information about the motion of the targets improves the tracking results, a single motion model may not capture the complex dynamic of the targets. This is a common situation with pedestrians, as each person moves in its own way, making tracking a difficult task. In this paper, this problem is faced by using a proposal based on the Interacting Multiple Model (IMM) and implemented in a Bayesian scheme through a particle filter. The core of this approach is to leave the filter choose the motion model that fits better the motion of the targets. The algorithm is evaluated, under several combinations of motion models, with middle-dense crowded scenes from the PETS 2009 dataset.

The aim of this work is to provide a fast approach for monocular SLAM initialization by constructing an initial 3-D map with interest points that are susceptible to be automatically tracked. Interest points' depth is inferred by means of a linear regression model, which estimates depth on the basis of local image appearance. Our contributions are: (1) a new scheme for learning and predicting associations between depth and local image appearance using RGB-D data; and (2) the use of this scheme for the initialization of state-of-the-art visual SLAM systems from a single image frame. To the best of our knowledge, this is the first attempt to automatically initialize a SLAM system by associating depth to sensor features through machine learning techniques. We performed a series of tests by making use of the celebrated PTAM system and obtained very promising results. We show successful one-shot initialization examples accomplished by applying our proposed approach to unstructured scene environments.

We present a novel method for multi-object tracking that tracks target both in the video streams and in a reference ground frame. This allows to remove ambiguities created by occlusions in one view. Our system takes as a base a recently proposed collaborative scheme and makes it handle multiple targets. We use a fast, simple solution for data association in the ground plane based on principal axis and a partly joint probabilistic model with MCMC sampling to ensure that tracked targets are kept separated whenever groups of targets appear. Results are presented on several popular databases of multi-camera, multi-target videos.

This paper presents a new approach for tracking objects in complex situations such as people in a crowd or players on a soccer field. Each object in the image is represented by several interest points (IPs). These IPs are obtained using a color version of the Harris IP detector. Each IP is characterized by the local appearance (chromatic first-order local jet) of the object and by geometric parameters. We track objects by matching IPs from image to image based on the Mahalanobis distance. The approach is robust to occlusion. Performance is illustrated by some examples.

This article presents a modular architecture for multicamera tracking in the context of sports broadcasting. For each video stream, a geometrical module continuously performs the image-to-model homography estimation. A local-feature based tracking module tracks the players in each view. A supervisor module collects, associates and fuses the data provided by the tracking modules. The originality of the proposed system is three-fold. First, it allows to localize the targets on the ground with rotating and zooming cameras; second, it does not use background modeling techniques; and third, the local tracking can cope with severe occlusions. We present experimental results on raw TV-camera footage of a soccer game.

This article focuses on the field-to-image registration for TV sports-related image processing through basic geometrical reasoning. We propose a robust automatic line-based method that can cope with imprecisions in the line detection steps through a hypothesis generation/pruning paradigm where the number of hypotheses is reduced by using the vanishing points. The results are satisfying and illustrated with examples from sport.

We describe an important element of an automatic sport analysis system. This element continuously estimates the image-to-model homography from the video stream of a single camera. Here, we focus on the incremental image-to-image updating of the homography matrix, which greatly facilitates real-time operation. This updating relies on the automatic tracking of interest lines and points and on their use for robust homography estimation. Results on real video sequences are shown. 1

The interaction between man and machines has become an important topic for the robotic community as it can generalise the use of robots. For active H/R interaction scheme, the robot needs to detect human faces in its vicinity and then interpret canonical gestures of the tracked person assuming this interlocutor has been beforehand identified. In this context, we depict functions suitable to detect and recognise faces in video stream and then focus on face or hand tracking functions. An efficient colour segmentation based on a watershed on the skin-like coloured pixels is proposed. A new measurement model is proposed to take into account both shape and colour cues in the particle filter to track face or hand silhouettes in video stream. An extension of the basic condensation algorithm is proposed to achieve recognition of the current hand posture and automatic switching between multiple templates in the tracking loop. Results of tracking and recognition using are illustrated in the paper and show the process robustness in cluttered environments and in various light conditions. The limits of the method and future works are also discussed.

This article describes visual functions dedicated to the extraction and recognition of planar quadrangles detected from a single camera. Extraction is based on a relaxation scheme with constraints between image segments, while the characterization we propose allows recognition to be achieved from different view-points and viewing conditions. We defined and evaluated several metrics on this representation space - a correlation-based one and another one based on sets of interest points.

This article describes the integration in a whole navigation system of visual functions dedicated to the extraction and recognition of visual landmarks, i.e. planar quadrangles detected from a single camera. The extraction of these landmarks is based on a relaxation scheme, used to satisfy constraints between image segments. During the exploration of an indoor environment, according to the current topology, there are several ways to take advantage of these visual landmarks. Two representations are considered, relying both on laser data and or our visual landmarks system: a GVG-based model (generalized Voronoi graph) for corridors, and a composite stochastic map for open spaces. In the particular case of corridors, the corresponding modules - laser GVG construction and visual landmarks extraction and recognition - can cooperate to complete each other, as image processing can be enhanced by some structural knowledge about the scene, whereas the GVG is annotated, even as far as its edges are concerned, by qualitative visual information.

Presents vision functions needed on a mobile robot to deal with landmark-based navigation in buildings. Landmarks are planar, quadrangular surfaces, which must be distinguished from the background, typically a poster on a wall or a door-plate. In a first step, these landmarks are detected and their positions with respect to a global reference frame are learned; this learning step is supervised so that only the best landmarks are memorized, with an invariant representation based on a set of interest points. Then, when the robot looks for visible landmarks, the recognition procedure takes advantage of the partial Hausdorff distance to compare the landmark model and the detected quadrangles. The paper presents the landmark detection and recognition procedures, and discusses their performances.

This article describes the integration in a complete navigation system of an environment modeling method based on a Generalized Voronoi Graph (GVG), relying on laser data, on the one hand, and of a localization method based on monocular vision landmark learning and recognition framework, on the other hand. Such a system is intended to work in structured environments. It is shown that the two corresponding modules - laser GVG construction and visual landmarks learning and recognition - can cooperate to complete each other, as image processing can be enhanced by some structural knowledge about the scene, whereas the GVG is annotated, even as far as its edges are concerned, by qualitative visual information.

Describes the localization function integrated in a landmark-based navigation system. It relies on the use of planar landmarks (typically, posters) to localize the robot. It is based on two periodic processes running at different frequencies. One of them performs the poster tracking (based on the partial Hausdorff distance) and the active control of the camera. The other one runs on a lower frequency and localizes the robot thanks to the tracked landmarks, the positions of which have been learnt during an offline exploration step. The system has been embedded on our indoor Hilare mobile robot and works in real time. Experiments, illustrated in the paper, demonstrate the validity of the approach.