International audience

Studying pedestrian navigation in crowded environments is key to many applications, like crowd simulation for urban planning, but is challenging under real-world conditions due to the lack of experimental control. This motivates the use of virtual reality to manipulate variables and recreate complex social interactions. While research has mostly emphasized visual feedback, and eventually haptic rendering of collision, the integration of auditory cues in VR platforms, such as footstep sounds, remains overlooked. The present study investigates the role of adding footstep sounds in a VR collision-avoidance task. Twenty participants completed a within-subject experiment where a virtual pedestrian, initially visually occluded, crossed their path either with or without audible footsteps. Locomotor adjustments and gaze behavior were analyzed before and during the visual interaction. Results showed that footstep sounds induced earlier gaze orientation and more proactive trajectory adjustments from the participants, without affecting their walking speed. In contrast, the absence of sound led to more collisions and crossing order inversions, indicating reduced efficiency. These findings demonstrate that simple auditory cues can guide attention, facilitate anticipation, and improve interactions with virtual agents. Integrating sound into VR platforms is therefore essential to study pedestrian behavior and social navigation.

Our goal in this article is to propose a fast and easy to implement BRDF modeling method that provides both accurate and compact representations for all types of BRDF, i.e., isotropic or anisotropic. To achieve this objective, we use a Bayesian regression method with a Gaussian process prior which allows obtaining compact BRDF representations in a purely analytical way. For this purpose, we use a generalzed distance covariance kernel which is much better suited to BRDF features than the usual Gaussian kernel. To speed up the processing, we adapt this method to the specificities of BRDFs through an appropriate input data structure and distribution of observations so as to drastically reduce the problem dimensionality through an efficient factorization method. In this way, all calculations at both fitting and rendering steps are reduced to basic matrix products and the computation of a BRDF representation with our modeling method takes only a few seconds. Furthermore, rather than using a systematic approach as in state-of-the-art methods, the size and complexity of the BRDF representation can be adapted to the application requirements as regards the fitting accuracy and rendering constraints. Besides, our BRDF representation can be easily converted to spherical harmonics expansions, which allows easier integration in usual rendering algorithms. We also propose importance sampling methods derived from our BRDF modeling method that leads to fast and easy implementations. Experimental applications of our method to various types of isotropic and anisotropic BRDFs show that state-of-the-art methods can be outperformed in most cases by using a small set of observations for the regression.

Abstract Crowd authoring has mainly focused on generalised agent interactions such as collision avoidance and grouping. However, in society, people interact more intentionally with specific “sources” such as exhibits, or inspectors. Uncovering these interactions is essential for understanding and characterising social behaviours. We propose a model that learns from trajectories, the localised agent interactions imposed by the context of the object or agent source. Our model decomposes agent paths into sequential combinations of simple and understandable “core” behaviours, like approach, stop, and circle around, temporally dissecting source-centric trajectories into standardised movements. We train on pairs of trajectory-encoded images and their associated core behaviour combination. Given a set of trajectories around a specific source, our framework can be applied to build a behaviour distribution, summarising how people interact with the source type. The inferred distribution can then be sampled to generate diverse crowds of context-aware agents. We evaluate our model using collected ground-truth data and perform a case study that showcases the utility of this decomposition of context-aware interactions in other tasks, such as measuring behaviour similarity.

Video-based high-density crowd analysis and prediction has been a long-standing topic in computer vision. It is notoriously difficult due to, but not limited to, the lack of high-quality data and complex crowd dynamics. Consequently, it has been relatively under-studied. In this paper, we propose a new approach that aims to learn from in-the-wild videos, often with low quality where it is difficult to track individuals or count heads. The key novelty is a new physics prior to model crowd dynamics. We model high-density crowds as active matter, a continuum with active particles subject to stochastic forces, named ‘crowd material’. Our physics model is combined with neural networks, resulting in a neural stochastic differential equation system that can mimic complex crowd dynamics. Due to the lack of similar research, we adapt a range of existing methods which are close to ours for comparison. Through exhaustive evaluations, we show our model outperforms existing methods in analyzing and forecasting extremely high-density crowds. Furthermore, since our model is a continuous-time physics model, it can be used for simulation and analysis, providing strong interpretability. This is categorically different from most deep learning methods, which are discrete-time models and black-boxes.

Abstract We present a method for animating herds that automatically tunes a microscopic herd model based on a short video clip of real animals. Our method handles videos with dense herds, where individual animal motion cannot be separated out. Our contribution is a novel framework for extracting macroscopic herd behaviour from such video clips, and then deriving the microscopic agent parameters that best match this behaviour. To support this learning process, we extend standard agent models to provide a separation between leaders and followers, better match the occlusion and field‐of‐view limitations of real animals, support differentiable parameter optimization and improve authoring control. We validate the method by showing that once optimized, the social force and perception parameters of the resulting herd model are accurate enough to predict subsequent frames in the video, even for macroscopic properties not directly incorporated in the optimization process. Furthermore, the extracted herding characteristics can be applied to any terrain with a palette and region‐painting approach that generalizes to different herd sizes and leader trajectories. This enables the authoring of herd animations in new environments while preserving learned behaviour.

Avoiding moving obstacles in immersive environments requires adjustments in the walking trajectory and depends on the type of obstacle movement. Previous research studied the impact of speed and direction of motion but not much is known about how predictability of motion impacts human circumvention in terms of distance to the obstacle (proximity). In this paper, we investigate how participants navigate through VR collision avoidance scenarios with obstacles of varying motion characteristics in terms of speed and predictability. We introduce a novel concept of creating unpredictable motion using entropy calculations. We anticipated that higher entropy would increase the proximity distance which we measured with several metrics related to the distance from the obstacle and centre of the scene. We found a significant influence of motion speed and predictability on proximity-related metrics, with participants exhibiting a tendency to maintain larger distances in scenarios where obstacle speed and entropy were higher. We also outline two decision-making strategies for avoidance behaviour and investigate the factors that influence individuals’ selection of one strategy over the other.

We investigate a new task in human motion prediction, which is predicting motions under unexpected physical perturbation potentially involving multiple people. Compared with existing research, this task involves predicting less controlled, unpremeditated and pure reactive motions in response to external impact and how such motions can propagate through people. It brings new challenges such as data scarcity and predicting complex interactions. To this end, we propose a new method capitalizing differentiable physics and deep neural networks, leading to an explicit Latent Differentiable Physics (LDP) model. Through experiments, we demonstrate that LDP has high data efficiency, outstanding prediction accuracy, strong generalizability and good explainability. Since there is no similar research, a comprehensive comparison with 11 adapted baselines from several relevant domains is conducted, showing LDP outperforming existing research both quantitatively and qualitatively, improving prediction accuracy by as much as 70%, and demonstrating significantly stronger generalization.

No abstract available.

We propose a novel contact-aware method to synthesize highly-dense 3D crowds of animated characters. Existing methods animate crowds by, first, computing the 2D global motion approximating subjects as 2D particles and, then, introducing individual character motions without considering their surroundings. This creates the illusion of a 3D crowd, but, with density, characters frequently intersect each other since character-to-character contact is not modeled. We tackle this issue and propose a general method that considers any crowd animation and resolves existing residual collisions. To this end, we take a physics-based approach to model contacts between articulated characters. This enables the real-time synthesis of 3D high-density crowds with dozens of individuals that do not intersect each other, producing an unprecedented level of physical correctness in animations. Under the hood, we model each individual using a parametric human body incorporating a set of 3D proxies to approximate their volume. We then build a large system of articulated rigid bodies, and use an efficient physics-based approach to solve for individual body poses that do not collide with each other while maintaining the overall motion of the crowd. We first validate our approach objectively and quantitatively. We then explore relations between physical correctness and perceived realism based on an extensive user study that evaluates the relevance of solving contacts in dense crowds. Results demonstrate that our approach outperforms existing methods for crowd animation in terms of geometric accuracy and overall realism.

Gaze behavior of virtual characters in video games and virtual reality experiences is a key factor of realism and immersion. Indeed, gaze plays many roles when interacting with the environment; not only does it indicate what characters are looking at, but it also plays an important role in verbal and non-verbal behaviors and in making virtual characters alive. Automated computing of gaze behaviors is however a challenging problem, and to date none of the existing methods are capable of producing close-to-real results in an interactive context. We therefore propose a novel method that leverages recent advances in several distinct areas related to visual saliency, attention mechanisms, saccadic behavior modelling, and head-gaze animation techniques. Our approach articulates these advances to converge on a multi-map saliency-driven model which offers real-time realistic gaze behaviors for non-conversational characters, together with additional user-control over customizable features to compose a wide variety of results. We first evaluate the benefits of our approach through an objective evaluation that confronts our gaze simulation with ground truth data using an eye-tracking dataset specifically acquired for this purpose. We then rely on subjective evaluation to measure the level of realism of gaze animations generated by our method, in comparison with gaze animations captured from real actors. Our results show that our method generates gaze behaviors that cannot be distinguished from captured gaze animations. Overall, we believe that these results will open the way for more natural and intuitive design of realistic and coherent gaze animations for real-time applications.

Crowd data is a crucial element in the modeling of collective behaviors, and opens the way to simulation for their study or prediction. Given the difficulty of acquiring such data, virtual reality is useful for simplifying experimental processes and opening up new experimental opportunities. This comes at the cost of the need to assess the biases introduced by the use of this technology. Our paper is part of this effort, and investigates the effect of the graphical representation of a crowd on the behavior of a user immersed within. More specifically, we inspect the virtual navigation through virtual crowds, in terms of travel speeds and local navigation choices as a function of the visual representation of the virtual agents that make up the crowd (simple geometric model, anthropomorphic model or realistic model). Through an experiment in which we ask a user to navigate virtual crowds of varying densities, we show that the effect of the visual representation is limited, but that an anthropomorphic representation offers the best trade-off between computational complexity and ecological validity, even though a more realistic representation can be preferred when user behaviour is studied in more details. Our work leads to clear recommendations on the design of immersive simulations for the study of crowd behavior.

While data is vital to better understand and model interactions within human crowds, capturing real crowd motions is extremely challenging. Virtual Reality (VR) demonstrated its potential to help, by immersing users into either simulated virtual crowds based on autonomous agents, or within motion-capture-based crowds. In the latter case, users' own captured motion can be used to progressively extend the size of the crowd, a paradigm called Record-and-Replay (2R). However, both approaches demonstrated several limitations which impact the quality of the acquired crowd data. In this paper, we propose the new concept of contextual crowds to leverage both crowd simulation and the 2R paradigm towards more consistent crowd data. We evaluate two different strategies to implement it, namely a Replace-Record-Replay (3R) paradigm where users are initially immersed into a simulated crowd whose agents are successively replaced by the user's captured-data, and a Replace-Record-Replay-Responsive (4R) paradigm where the pre-recorded agents are additionally endowed with responsive capabilities. These two paradigms are evaluated through two real-world-based scenarios replicated in VR. Our results suggest that the behaviors observed in VR users with surrounding agents from the beginning of the recording process are made much more natural, enabling 3R or 4R paradigms to improve the consistency of captured crowd datasets.

Nonverbal communication is paramount in daily life, as well as in populated virtual reality (VR) environments. In this paper, we focused on gaze behaviour, which is key to initiate and drive social interactions. Previous work on photographs and on virtual agents showed the importance of gaze, even in the presence of multiple stimuli, by demonstrating the stare-in-the-crowd effect: humans detect faster and observe gazes directed towards them longer than the averted ones. While previous studies focused on static scenarios, which fail in representing the complexity of real-life social interactions, we propose to explore the stare-in-the-crowd effect in dynamic situations. To this end, we designed a within-subject experiment where 21 users navigated a virtual street through an idle or moving crowd of virtual agents. Agents’ gaze was manipulated to display averted, directed, or shifting gaze. We analysed the user’s gaze (fixations, dwell time) and locomotor behaviours (path decisions, proximity to agents) as well as their social anxiety. Results showed that the stare-in-the-crowd effect is preserved when walking through both types of crowd, and that social anxiety decreases gaze interaction time and affects proximity behaviours in case of agents with directed gazes. However, virtual agents’ gaze did not elicit significant changes on users’ locomotion. These findings highlight the importance of considering virtual agents’ gaze when creating VR environments, and open future work perspectives to better understand factors that would strengthen or decrease this effect at gaze and locomotor levels.

Our goal is to simulate how humans recover balance after external perturbation, e.g., being pushed. While different strategies can be adopted to achieve balance recovery, we particularly aim at replicating how humans combine the control of their support area with the control of their body movement to regain balance when it is necessary. We develop a physics-based approach to simulate balance recovery, with two main contributions to achieve our goal: a foot control technique to adjust the shape of a character’s support zone to the motion of its center of mass (CoM), and the dynamic control of the CoM to maintain its vertical projection in this same zone. We also calibrate the simulation by optimisation, before validating our results against experimental data.

Crowd simulation is important for video-games design, since it enables to populate virtual worlds with autonomous avatars that navigate in a human-like manner. Reinforcement learning has shown great potential in simulating virtual crowds, but the design of the reward function is critical to achieving effective and efficient results. In this work, we explore the design of reward functions for reinforcement learning-based crowd simulation. We provide theoretical insights on the validity of certain reward functions according to their analytical properties, and evaluate them empirically using a range of scenarios, using the energy efficiency as the metric. Our experiments show that directly minimizing the energy usage is a viable strategy as long as it is paired with an appropriately scaled guiding potential, and enable us to study the impact of the different reward components on the behavior of the simulated crowd. Our findings can inform the development of new crowd simulation techniques, and contribute to the wider study of human-like navigation.

Simulating crowds with realistic behaviors is a difficult but very important task for a variety of applications. Quantifying how a person balances between different conflicting criteria such as goal seeking, collision avoidance and moving within a group is not intuitive, especially if we consider that behaviors differ largely between people. Inspired by recent advances in Deep Reinforcement Learning, we propose Guided REinforcement Learning (GREIL) Crowds, a method that learns a model for pedestrian behaviors which is guided by reference crowd data. The model successfully captures behaviors such as goal seeking, being part of consistent groups without the need to define explicit relationships and wandering around seemingly without a specific purpose. Two fundamental concepts are important in achieving these results: (a) the per agent state representation and (b) the reward function. The agent state is a temporal representation of the situation around each agent. The reward function is based on the idea that people try to move in situations/states in which they feel comfortable in. Therefore, in order for agents to stay in a comfortable state space, we first obtain a distribution of states extracted from real crowd data; then we evaluate states based on how much of an outlier they are compared to such a distribution. We demonstrate that our system can capture and simulate many complex and subtle crowd interactions in varied scenarios. Additionally, the proposed method generalizes to unseen situations, generates consistent behaviors and does not suffer from the limitations of other data-driven and reinforcement learning approaches.

Analyzing motion data is a critical step when building meaningful locomotive motion datasets. This can be done by labeling motion capture data and inspecting it, through a planned motion capture session or by carefully selecting locomotion clips from a public dataset. These analyses, however, have no clear definition of coverage, making it harder to diagnose when something goes wrong, such as a virtual character not being able to perform an action or not moving at a given speed. This issue is compounded by the large amount of information present in motion capture data, which poses a challenge when trying to interpret it. This work provides a visualization and an optimization method to streamline the process of crafting locomotive motion datasets. It provides a more grounded approach towards locomotive motion analysis by calculating different quality metrics, such as: demarcating coverage in terms of both linear and angular speeds, frame use frequency in each animation clip, deviation from the planned path, number of transitions, number of used vs. unused animations and transition cost.

Nonverbal cues are paramount in real-world interactions. Among these cues, gaze has received much attention in the literature. In particular, previous work has shown a search asymmetry between directed and averted gaze towards the observer using photographic stimuli, with faster detection and longer fixation towards directed gaze by the observer. This is known as the stare-in-the-crowd effect. In this study, we investigate whether stare-in-the crowd effect is preserved in Virtual Reality (VR). To this end, we designed a within-subject experiment where 30 human users were immersed in a virtual environment in front of an audience of 11 virtual agents following 4 different gaze behaviours. We analysed the user’s gaze behaviour when observing the audience, computing fixations and dwell time. We also collected the users’ social anxiety score using a post-experiment questionnaire to control for some potential influencing factors. Results show that the stare-in-the-crowd effect is preserved in VR, as demonstrated by the significant differences between gaze behaviours, similarly to what was found in previous studies using photographic stimuli. Additionally, we found a negative correlation between dwell time towards directed gazes and users’ social anxiety scores. Such results are encouraging for the development of expressive and reactive virtual humans, which can be animated to express natural interactive behaviour.

Abstract The real‐time simulation of human crowds has many applications. In a typical crowd simulation, each person ('agent') in the crowd moves towards a goal while adhering to local constraints. Many algorithms exist for specific local ‘steering’ tasks such as collision avoidance or group behavior. However, these do not easily extend to completely new types of behavior, such as circling around another agent or hiding behind an obstacle. They also tend to focus purely on an agent's velocity without explicitly controlling its orientation. This paper presents a novel sketch‐based method for modelling and simulating many steering behaviors for agents in a crowd. Central to this is the concept of aninteraction field(IF): a vector field that describes the velocities or orientations that agents should use around a given ‘source’ agent or obstacle. An IF can also change dynamically according to parameters, such as the walking speed of the source agent. IFs can be easily combined with other aspects of crowd simulation, such as collision avoidance. Using an implementation of IFs in a real‐time crowd simulation framework, we demonstrate the capabilities of IFs in various scenarios. This includes game‐like scenarios where the crowd responds to a user‐controlled avatar. We also present an interactive tool that computes an IF based on input sketches. This IF editor lets users intuitively and quickly design new types of behavior, without the need for programming extra behavioral rules. We thoroughly evaluate the efficacy of the IF editor through a user study, which demonstrates that our method enables non‐expert users to easily enrich any agent‐based crowd simulation with new agent interactions.

Abstract Simulating crowds requires controlling a very large number of trajectories of characters and is usually performed using crowd steering algorithms. The question of choosing the right algorithm with the right parameter values is of crucial importance given the large impact on the quality of results. In this paper, we study the performance of a number of steering policies (i.e., simulation algorithm and its parameters) in a variety of contexts, resorting to an existing quality function able to automatically evaluate simulation results. This analysis allows us to map contexts to the performance of steering policies. Based on this mapping, we demonstrate that distributing the best performing policies among characters improves the resulting simulations. Furthermore, we also propose a solution to dynamically adjust the policies, for each agent independently and while the simulation is running, based on the local context each agent is currently in. We demonstrate significant improvements of simulation results compared to previous work that would optimize parameters once for the whole simulation, or pick an optimized, but unique and static, policy for a given global simulation context.

Abstract Reinforcement Learning is an area of Machine Learning focused on how agents can be trained to make sequential decisions, and achieve a particular goal within an arbitrary environment. While learning, they repeatedly take actions based on their observation of the environment, and receive appropriate rewards which define the objective. This experience is then used to progressively improve the policy controlling the agent's behavior, typically represented by a neural network. This trained module can then be reused for similar problems, which makes this approach promising for the animation of autonomous, yet reactive characters in simulators, video games or virtual reality environments. This paper surveys the modern Deep Reinforcement Learning methods and discusses their possible applications in Character Animation, from skeletal control of a single, physically‐based character to navigation controllers for individual agents and virtual crowds. It also describes the practical side of training DRL systems, comparing the different frameworks available to build such agents.

Abstract Recent advancements in crowd simulation unravel a wide range of functionalities for virtual agents, delivering highly‐realistic, natural virtual crowds. Such systems are of particular importance to a variety of applications in fields such as: entertainment (e.g., movies, computer games); architectural and urban planning; and simulations for sports and training. However, providing their capabilities to untrained users necessitates the development of authoring frameworks. Authoring virtual crowds is a complex and multi‐level task, varying from assuming control and assisting users to realise their creative intents, to delivering intuitive and easy to use interfaces, facilitating such control. In this paper, we present a categorisation of the authorable crowd simulation components, ranging from high‐level behaviours and path‐planning to local movements, as well as animation and visualisation. We provide a review of the most relevant methods in each area, emphasising the amount and nature of influence that the users have over the final result. Moreover, we discuss the currently available authoring tools (e.g., graphical user interfaces, drag‐and‐drop), identifying the trends of early and recent work. Finally, we suggest promising directions for future research that mainly stem from the rise of learning‐based methods, and the need for a unified authoring framework.

When two streams of pedestrians cross at an angle, striped patterns spontaneously emerge as a result of local pedestrian interactions. This clear case of self-organized pattern formation remains to be elucidated. In counterflows, with a crossing angle of 180°, alternating lanes of traffic are commonly observed moving in opposite directions, whereas in crossing flows at an angle of 90°, diagonal stripes have been reported. Naka (1977) hypothesized that stripe orientation is perpendicular to the bisector of the crossing angle. However, studies of crossing flows at acute and obtuse angles remain underdeveloped. We tested the bisector hypothesis in experiments on small groups (18-19 participants each) crossing at seven angles (30° intervals), and analyzed the geometric properties of stripes. We present two novel computational methods for analyzing striped patterns in pedestrian data: (i) an edge-cutting algorithm, which detects the dynamic formation of stripes and allows us to measure local properties of individual stripes; and (ii) a pattern-matching technique, based on the Gabor function, which allows us to estimate global properties (orientation and wavelength) of the striped pattern at a time T. We find an invariant property: stripes in the two groups are parallel and perpendicular to the bisector at all crossing angles. In contrast, other properties depend on the crossing angle: stripe spacing (wavelength), stripe size (number of pedestrians per stripe), and crossing time all decrease as the crossing angle increases from 30° to 180°, whereas the number of stripes increases with crossing angle. We also observe that the width of individual stripes is dynamically squeezed as the two groups cross each other. The findings thus support the bisector hypothesis at a wide range of crossing angles, although the theoretical reasons for this invariant remain unclear. The present results provide empirical constraints on theoretical studies and computational models of crossing flows.

Virtual reality (VR) is a valuable experimental tool for studying human movement, including the analysis of interactions during locomotion tasks for developing crowd simulation algorithms. However, these studies are generally limited to distant interactions in crowds, due to the difficulty of rendering realistic sensations of collisions in VR. In this article, we explore the use of wearable haptics to render contacts during virtual crowd navigation. We focus on the behavioral changes occurring with or without haptic rendering during a navigation task in a dense crowd, as well as on potential after-effects introduced by the use haptic rendering. Our objective is to provide recommendations for designing VR setup to study crowd navigation behavior. To the end, we designed an experiment (N=23) where participants navigated in a crowded virtual train station without, then with, and then again without haptic feedback of their collisions with virtual characters. Results show that providing haptic feedback improved the overall realism of the interaction, as participants more actively avoided collisions. We also noticed a significant after-effect in the users' behavior when haptic rendering was once again disabled in the third part of the experiment. Nonetheless, haptic feedback did not have any significant impact on the users' sense of presence and embodiment.

Crowd motion data is fundamental for understanding and simulating realistic crowd behaviours. Such data is usually collected through controlled experiments to ensure that both desired individual interactions and collective behaviours can be observed. It is however scarce, due to ethical concerns and logistical difficulties involved in its gathering, and only covers a few typical crowd scenarios. In this work, we propose and evaluate a novel Virtual Reality based approach lifting the limitations of real-world experiments for the acquisition of crowd motion data. Our approach immerses a single user in virtual scenarios where he/she successively acts each crowd member. By recording the past trajectories and body movements of the user, and displaying them on virtual characters, the user progressively builds the overall crowd behaviour by him/herself. We validate the feasibility of our approach by replicating three real experiments, and compare both the resulting emergent phenomena and the individual interactions to existing real datasets. Our results suggest that realistic collective behaviours can naturally emerge from virtual crowd data generated using our approach, even though the variety in behaviours is lower than in real situations. These results provide valuable insights to the building of virtual crowd experiences, and reveal key directions for further improvements.

Tracking humans in crowded video sequences is an important constituent of visual scene understanding. Increasing crowd density challenges visibility of humans, limiting the scalability of existing pedestrian trackers to higher crowd densities. For that reason, we propose to revitalize head tracking with Crowd of Heads Dataset (CroHD), consisting of 9 sequences of 11,463 frames with over 2,276,838 heads and 5,230 tracks annotated in diverse scenes. For evaluation, we proposed a new metric, IDEucl, to measure an algorithm’s efficacy in preserving a unique identity for the longest stretch in image coordinate space, thus building a correspondence between pedestrian crowd motion and the performance of a tracking algorithm. Moreover, we also propose a new head detector, HeadHunter, which is designed for small head detection in crowded scenes. We extend HeadHunter with a Particle Filter and a color histogram based re-identification module for head tracking. To establish this as a strong baseline, we compare our tracker with existing state-of-the-art pedestrian trackers on CroHD and demonstrate superiority, especially in identity preserving tracking metrics. With a light-weight head detector and a tracker which is efficient at identity preservation, we believe our contributions will serve useful in advancement of pedestrian tracking in dense crowds. We make our dataset, code and models publicly available at https://project.inria.fr/crowdscience/project/dense-crowd-head-tracking/.

The evaluation of robot capabilities to navigate human crowds is essential to conceive new robots intended to operate in public spaces. This paper initiates the development of a benchmark tool to evaluate such capabilities; our long term vision is to provide the community with a simulation tool that generates virtual crowded environment to test robots, to establish standard scenarios and metrics to evaluate navigation techniques in terms of safety and efficiency, and thus, to install new methods to benchmarking robots’ crowd navigation capabilities. This paper presents the architecture of the simulation tools, introduces first scenarios and evaluation metrics, as well as early results to demonstrate that our solution is relevant to be used as a benchmark tool.

Non-verbal communication body cues are paramount to interact. In this preliminary work, we explore ways to let Intelligent Virtual Agents (IVAs) simulating nonverbal communication capabilities. We propose an approach to control IVAs' reactive behaviour from the analysis of other agents' apparent motions, in a situation of "observed" IVAs that act and "observers" that react. For that, first a viewpoint-driven analysis of the observed agent's motion is done, and then a synthesis of this analysis induces the observers' reaction.

Virtual human crowds are regularly featured in movies and video games. With a large number of virtual characters each behaving in their own way, spectacular scenes can be produced. The more diverse the characters and their behaviors are, the more realistic the virtual crowd is expected to be perceived. Hence, creating virtual crowds is a trade-off between the cost associated with acquiring more diverse assets, namely more virtual characters with their animations, and achieving better realism. In this paper, our focus is on the perceived variety in virtual crowd character motions. We present an experiment exploring whether observers are able to identify virtual crowds including motion clones in the case of large-scale crowds (from 250 to 1000 characters). As it is not possible to acquire individual motions for such numbers of characters, we rely on a state-of-the-art motion variation approach to synthesize unique variations of existing examples for each character in the crowd. Participants then compared pairs of videos, where each character was animated either with a unique motion or using a subset of these motions. Our results show that virtual crowds with more than two motions (one per gender) were perceptually equivalent, regardless of their size. We believe these findings can help create efficient crowd applications, and are an additional step into a broader understanding of the perception of motion variety.

Abstract The real‐time simulation of human crowds has many applications. Simulating how the people in a crowd move through an environment is an active and ever‐growing research topic. Most research focuses on microscopic (or ‘agent‐based’) crowd‐simulation methods that model the behavior of each individual person, from which collective behavior can then emerge. This state‐of‐the‐art report analyzes how the research on microscopic crowd simulation has advanced since the year 2010. We focus on the most popular research area within the microscopic paradigm, which is local navigation, and most notably collision avoidance between agents. We discuss the four most popular categories of algorithms in this area (force‐based, velocity‐based, vision‐based, and data‐driven) that have either emerged or grown in the last decade. We also analyze the conceptual and computational (dis)advantages of each category. Next, we extend the discussion to other types of behavior or navigation (such as group behavior and the combination with path planning), and we review work on evaluating the quality of simulations. Based on the observed advancements in the 2010s, we conclude by predicting how the research area of microscopic crowd simulation will evolve in the future. Overall, we expect a significant growth in the area of data‐driven and learning‐based agent navigation, and we expect an increasing number of methods that re‐group multiple ‘levels’ of behavior into one principle. Furthermore, we observe a clear need for new ways to analyze (real or simulated) crowd behavior, which is important for quantifying the realism of a simulation and for choosing the right algorithms at the right time.

Synthesizing motions that look realistic and diverse is a challenging task in animation. Therefore, a few generic walking motions are typically used when creating crowds of walking virtual characters, leading to a lack of variations as motions are not necessarily adapted to each and every virtual character’s characteristics. While some attempts have been made to create variations, it appears necessary to identify the relevant parameters that influence users’ perception of such variations to keep a good trade-off between computational costs and realism. In this paper, we therefore investigate the ability of viewers to identify an invariant parameter of human walking named the Walk Ratio (step length to step frequency ratio), which was shown to be specific to each individual and constant for different speeds, but which has never been used to drive animations of virtual characters. To this end, we captured 2 female and 2 male actors walking at different freely chosen speeds, as well as at different combinations of step frequency and step length. We then performed a perceptual study to identify the Walk Ratio that was perceived as the most natural for each actor when animating a virtual character, and compared it to the Walk Ratio freely chosen by the actor during the motion capture session. We found that Walk Ratios chosen by observers were in the range of Walk Ratios measured in the literature, and that participants perceived differences between the Walk Ratios of animated male and female characters, as evidenced in the biomechanical literature. Our results provide new considerations to drive the animation of walking virtual characters using the Walk Ratio as a parameter, and might provide animators with novel means to control the walking speed of characters through simple parameters while retaining the naturalness of the locomotion.

In highly dense crowds of humans, collisions between people occur often. It is common to simulate such a crowd as one fluid-like entity (macroscopic), and not as a set of individuals (microscopic, agent-based). Agent-based simulations are preferred for lower densities because they preserve the properties of individual people. However, their collision handling is too simplistic for extreme-density crowds. Therefore, neither paradigm is ideal for all possible densities.

To simulate the low-level (‘microscopic’) behavior of human crowds, a local navigation algorithm computes how a single person (‘agent’) should move based on its surroundings. Many algorithms for this purpose have been proposed, each using different principles and implementation details that are difficult to compare.

When we are walking in crowds, we mainly use visual information to avoid collisions with other pedestrians. Thus, gaze activity should be considered to better understand interactions between people in a crowd. In this work, we use Virtual Reality (VR) to facilitate motion and gaze tracking, as well as to accurately control experimental conditions, in order to study the effect of crowd density on eye-gaze behavior. Our motivation is to better understand how interaction neighborhood (i.e., the subset of people actually influencing one’s locomotion trajectory) changes with density. To this end, we designed two experiments. The first one evaluates the biases introduced by the use of VR on the visual activity when walking among people, by comparing eye-gaze activity while walking in a real and virtual street. We then designed a second experiment where participants walked in a virtual street with different levels of pedestrian density. We demonstrate that gaze fixations are performed at the same frequency despite increases in pedestrian density, while the eyes scan a narrower portion of the street. These results suggest that in such situations walkers focus more on people in front and closer to them. These results provide valuable insights regarding eye-gaze activity during interactions between people in a crowd, and suggest new recommendations in designing more realistic crowd simulations.

This paper explores the use of Virtual Reality (VR) to study humanrobot interactions during navigation tasks by both immersing a user and a robot in a shared virtual spaces. VR combines the advantages of being safe (as robots and humans interacting by the means of VR but can physically be in remote places) and ecological (realistic environments are perceived by the robot and the human, and natural behaviors can be observed). Nevertheless, VR can introduce perceptual biases in the interaction and affect in some ways the observed behaviors, which can be problematic when used to acquire experimental data. In our case, not only human perception is concerned, but also the one of the robot which requires to be simulated to perceive the VR world. Thus, the contribution of this paper is twofold. It first provides a technical solution to perform human robot interactions in navigation tasks through VR: we describe how we combine motion tracking, VR devices, as well as robot sensors simulation algorithms to immerse together a human and a robot in a shared virtual space. We then assess a simple interaction task that we replicate in real and in virtual conditions to perform a first estimation of the importance of the biases introduced by the use of VR on both a Human and a robot. Our conclusions are in favor of using VR to study human-robot interactions, and we are developing directions for future work.

In human interaction, people will keep different distances from each other depending on their gender. For example, males will stand further away from males and closer to females. Previous studies in virtual reality (VR), where people were interacting with virtual humans, showed a similar result. However, many other variables influence proximity, such as appearance characteristics of the virtual character (e.g., attractiveness). Our study focuses on proximity to virtual walkers, where gender could be recognised from motion only, since previous studies using point-light displays found walking motion is rich in gender cues. In our experiment, a walking wooden mannequin approached the participant embodied in a virtual avatar using the HTC Vive Pro HMD and controllers. The mannequin animation was motion captured from several male and female actors and each motion was displayed individually on the character. Participants used the controller to stop the approaching mannequin when they felt it was uncomfortably close to them. Based on previous work, we hypothesised that proximity will be affected by the gender of the character, but unlike previous research, the gender in our experiment could only be determined from character’s motion. We also expected differences in proximity according to the gender of the participant. We additionally expected some motions to be rated more attractive than others and that attractive motions would reduce the proximity measure. Our results show support for the last two assumptions, but no difference in proximity was found according to the gender of the character’s motion. Our findings have implications for the design of virtual characters in interactive virtual environments.

This paper presents a novel data-driven crowd simulation method that can mimic the observed traffic of pedestrians in a given environment. Given a set of observed trajectories, we use a recent form of neural networks, Generative Adversarial Networks (GANs), to learn the properties of this set and generate new trajectories with similar properties. We define a way for simulated pedestrians (agents) to follow such a trajectory while handling local collision avoidance. As such, the system can generate a crowd that behaves similarly to observations, while still enabling real-time interactions between agents. Via experiments with real-world data, we show that our simulated trajectories preserve the statistical properties of their input. Our method simulates crowds in real time that resemble existing crowds, while also allowing insertion of extra agents, combination with other simulation methods, and user interaction.

This paper introduces the use of numerical optical flow (OF) in vision-based steering techniques - that control characters locomotion trajectories by using a simulation of their visual perception. In contrast with synthetic OF that was previously used, numerical OF is sensitive to the contrast of objects, and provides, for example, uncertain results in dark areas. Thus, we here propose a locomotion control technique which is robust to such uncertainty: dark areas in the scene are processed as obstacles, that however may be traversed in case of necessity. As demonstrated in various scenarios, this tends to make character avoiding darkest areas, or traversing them more carefully, as it can be observed for real humans.

No abstract available.

We present a novel topology-driven method for improving the navigation of agents in virtual environments. In agent-based crowd simulations, the combination of global path planning and local collision avoidance can cause conflicts and undesired motion. These conflicts are related to the decisions to pass obstacles or agents on certain sides. In this paper, we define an agent’s navigation behavior as a topological strategy amidst obstacles and other agents. We show how to extract such a strategy from a global path and from a local velocity. Next, we propose a simulation framework that computes these strategies for path planning, path following, and collision avoidance. By detecting conflicts between strategies, we can decide reliably when and how an agent should re-plan an alternative path. As such, this work bridges a long-existing gap between global and local planning. Experiments show that our method can improve the behavior of agents while preserving real-time performance. It can be applied to many agent-based simulations, regardless of their specific navigation algorithms. The strategy concept is also suitable for explicitly sending agents in particular directions.

Recent works in the domain of Human-Robot Motion (HRM) attempted to plan collision avoidance behavior that accounts for cooperation between agents. Cooperative collision avoidance between humans and robots should be conducted under several factors such as speed, heading and also human attention and intention. Based on some of these factors, people decide their crossing order during collision avoidance. However, whenever situations arise in which the choice crossing order is not consistent for people, the robot is forced to account for the possibility that both agents will assume the same role i.e. a decision detrimental to collision avoidance. In our work we evaluate the boundary that separates the decision to avoid collision as first or last crosser. Approximating the uncertainty around this boundary allows our collision avoidance strategy to address this problem based on the insight that the robot should plan its collision avoidance motion in such a way that, even if agents, at first, incorrectly choose the same crossing order, they would be able to unambiguously perceive their crossing order on their following collision avoidance action.

Simulating realistic interactions between virtual characters has been of interest to research communities for years, and is particularly important to automatically populate virtual environments. This problem requires to accurately understand and model how humans interact, which can be difficult to assess. In this context, Virtual Reality (VR) is a powerful tool to study human behaviour, especially as it allows assessing conditions which are both ecological and controlled. While VR was shown to allow realistic collision avoidance adaptations, in the frame of the ecological theory of perception and action, interactions between walkers can not solely be characterized through motion adaptations but also through the perception processes involved in such interactions. The objective of this paper is therefore to evaluate how different VR setups influence gaze behaviour during collision avoidance tasks between walkers. To this end, we designed an experiment involving a collision avoidance task between a participant and another walker (real confederate or virtual character). During this task, we compared both the partici-pant`s locomotion and gaze behaviour in a real environment and the same situation in different VR setups (including a CAVE, a screen and a Head-Mounted Display). Our results show that even if some quantitative differences exist, gaze behaviour is qualitatively similar between VR and real conditions. Especially, gaze behaviour in VR setups including a HMD is more in line with the real situation than the other setups. Furthermore, the outcome on motion adaptations confirms previous work, where collision avoidance behaviour is qualitatively similar in VR and real conditions. In conclusion, our results show that VR has potential for qualitative analysis of locomotion and gaze behaviour during collision avoidance. This opens perspectives in the design of new experiments to better understand human behaviour, in order to design more realistic virtual humans.

Abstract Steering and navigation are important components of character animation systems to enable them to autonomously move in their environment. In this work, we propose a synthetic vision model that uses visual features to steer agents through dynamic environments. Our agents perceive optical flow resulting from their relative motion with the objects of the environment. The optical flow is then segmented and processed to extract visual features such as the focus of expansion and time‐to‐collision. Then, we establish the relations between these visual features and the agent motion, and use them to design a set of control functions which allow characters to perform object‐dependent tasks, such as following, avoiding and reaching. Control functions are then combined to let characters perform more complex navigation tasks in dynamic environments, such as reaching a goal while avoiding multiple obstacles. Agent's motion is achieved by local minimization of these functions. We demonstrate the efficiency of our approach through a number of scenarios. Our work sets the basis for building a character animation system which imitates human sensorimotor actions. It opens new perspectives to achieve realistic simulation of human characters taking into account perceptual factors, such as the lighting conditions of the environment.

Abstract Animating herds of animals while achieving both convincing global shapes and plausible distributions within the herd is difficult, using simulation methods. In this work, we allow users to rely on photos of real herds, which are widely available, for keyframing their animation. More precisely, we learn global and local distribution features in each photo of the input set (which may depict different numbers of animals) and transfer them to the group of animals to be animated, thanks to a new statistical learning method enabling to analyze distributions of ellipses, as well as their density and orientation fields. The animated herd reconstructs the desired distribution at each keyframe while avoiding obstacles. As our results show, our method offers both high‐level user control and help toward realism, enabling to easily author herd animations.

Recent works in the area of human robot motion showed that behaving in a human-like manner allows a robot to reduce global cognitive effort for people in the environment. Given that collision avoidance situations between people are solved cooperatively, this work models the manner in which this cooperation is done so that a robot can replicate their behavior. To that end, hundreds of situations where two walkers have crossing trajectories were analyzed. Based on these human trajectories involving a collision avoidance task, we determined how total effort is shared between each walker depending on several factors of the interaction such as crossing angle, time to collision and speed. To validate our approach, a proof of concept is integrated into ROS with Reciprocal Velocity Objects (RVO) in order to distribute collision avoidance effort in a human-like way.

This paper presents a study performed in virtual reality on the effect of gaze interception during collision avoidance between two walkers. In such a situation, mutual gaze can be considered as a form of nonverbal communication. Additionally, gaze is believed to detail future path intentions and to be part of the nonverbal negotiation to achieve avoidance collaboratively. We considered an avoidance task between a real subject and a virtual human character and studied the influence of the character's gaze direction on the avoidance behaviour of the participant. Virtual reality provided an accurate control of the situation: seventeen participants were immersed in a virtual environment, instructed to navigate across a virtual space using a joystick and to avoid a virtual character that would appear from either side. The character would either gaze or not towards the participant. Further, the character would either perform or not a reciprocal adaptation of its trajectory to avoid a potential collision with the participant. The findings of this paper were that during an orthogonal collision avoidance task, gaze behaviour did not influence the collision avoidance behaviour of the participants. Further, the addition of reciprocal collision avoidance with gaze did not modify the collision behaviour of participants. These results suggest that for the duration of interaction in such a task, body motion cues were sufficient for coordination and regulation. We discuss the possible exploitation of these results to improve the design of virtual characters for populated virtual environments and user interaction.

Synthesizing motions for legged characters in arbitrary environments is a long-standing problem that has recently received a lot of attention from the computer graphics community. We tackle this problem with a procedural approach that is generic, fully automatic, and independent from motion capture data. The main contribution of this article is a point-mass-model-based method to synthesize Center Of Mass trajectories. These trajectories are then used to generate the whole-body motion of the character. The use of a point mass model results in physically inconsistent motions and joint limit violations when mapped back to a full- body motion. We mitigate these issues through the use of a novel formulation of the kinematic constraints that allows us to generate a quasi-static Center Of Mass trajectory in a way that is both user-friendly and computationally efficient. We also show that the quasi-static constraint can be relaxed to generate motions usable for computer animation at the cost of a moderate violation of the dynamic constraints. Our method was integrated in our open-source contact planner and tested with different scenarios—some never addressed before—featuring legged characters performing non-gaited motions in cluttered environments. The computational efficiency of our trajectory generation algorithm (under one ms to compute one second of trajectory) enables us to synthesize motions in a few seconds, one order of magnitude faster than state-of-the-art methods. Although our method is empirically able to synthesize collision-free motions, the formal handling of environmental constraints is not part of the proposed method and left for future work.

We present a contact planner for complex legged locomotion tasks: standing up, climbing stairs using a handrail, crossing rubble, and getting out of a car. The need for such a planner was shown at the DARPA Robotics Challenge, where such behaviors could not be demonstrated (except for egress). Current planners suffer from their prohibitive algorithmic complexity because they deploy a tree of robot configurations projected in contact with the environment. We tackle this issue by introducing a reduction property: the reachability condition. This condition defines a geometric approximation of the contact manifold, which is of low dimension, presents a Cartesian topology, and can be efficiently sampled and explored. The hard contact planning problem can then be decomposed into two subproblems: first, we plan a path for the root without considering the whole-body configuration, using a sampling-based algorithm; then, we generate a discrete sequence of whole-body configurations in static equilibrium along this path, using a deterministic contact-selection algorithm. The reduction breaks the algorithm complexity encountered in previous works, resulting in the first interactive implementation of a contact planner (open source). While no contact planner has yet been proposed with theoretical completeness, we empirically show the interest of our framework: in a few seconds, with high success rates, we generate complex contact plans for various scenarios and two robots: HRP-2 and HyQ. These plans are validated in dynamic simulations or on the real HRP-2 robot.

Daily activities require agents to interact with each other, such as during collision avoidance. The nature of visual information that is used for a collision free interaction requires further understanding. We aim to manipulate the nature of visual information in two forms, global and local information appearances. Sixteen healthy participants navigated towards a target in an immersive computer-assisted virtual environment (CAVE) using a joystick. A moving passive obstacle crossed the participant's trajectory perpendicularly at various pre-defined risks of collision distances. The obstacle was presented with one of five virtual appearances, associated to global motion cues (i.e., a cylinder or a sphere), or local motion cues (i.e., only the legs or the trunk). A full body virtual walker, showing both local and global motion cues, used as a reference condition. The final crossing distance was affected by the global motion appearances, however, appearance had no qualitative effect on motion adaptations. These findings contribute towards further understanding what information people use when interacting with others.

Navigating in virtual environments requires using some locomotion interfaces, especially when the dimensions of the environment exceed the ones of the Virtual Reality system. Locomotion interfaces induce some biases both in the perception of the self-motion or in the formation of virtual locomotion trajectories. These biases have been mostly evaluated in the context of static environments, and studies need to be revisited in the new context of populated environments where users interact with virtual characters. We focus on a situation of collision avoidance between a real participant and a virtual character, and compared it to previous studies on real walkers. Our results show that, as in reality, the risk of future collision is accurately anticipated by participants, however with delay. We also show that collision avoidance trajectories formed in VR have common properties with real ones, with some quantitative differences in avoidance distances. More generally, our evaluation demonstrates that reliable results can be obtained for qualitative analysis of small scale interactions in VR. We discuss these results in the perspective of a VR platform for large scale interaction applications, such as in a crowd, for which real data are difficult to gather.

Abstract When navigating in crowds, humans are able to move efficiently between people. They look ahead to know which path would reduce the complexity of their interactions with others. Current navigation systems for virtual agents consider long‐term planning to find a path in the static environment and short‐term reactions to avoid collisions with close obstacles. Recently some mid‐term considerations have been added to avoid high density areas. However, there is no mid‐term planning among static and dynamic obstacles that would enable the agent to look ahead and avoid difficult paths or find easy ones as humans do. In this paper, we present a system for such mid‐term planning. This system is added to the navigation process between pathfinding and local avoidance to improve the navigation of virtual agents. We show the capacities of such a system using several case studies. Finally we use an energy criterion to compare trajectories computed with and without the mid‐term planning.

Most recent crowd simulation algorithms equip agents with a synthetic vision component for steering. They offer promising perspectives through a more realistic simulation of the way humans navigate according to their perception of the surrounding environment. In this paper, we propose a new perception/motion loop to steering agents along collision free trajectories that significantly improves the quality of vision-based crowd simulators. In contrast with solutions where agents avoid collisions in a purely reactive (binary) way, we suggest exploring the full range of possible adaptations and retaining the locally optimal one. To this end, we introduce a cost function, based on perceptual variables, which estimates an agent's situation considering both the risks of future collision and a desired destination. We then compute the partial derivatives of that function with respect to all possible motion adaptations. The agent then adapts its motion by following the gradient. This paper has thus two main contributions: the definition of a general purpose control scheme for steering synthetic vision-based agents; and the proposition of cost functions for evaluating the perceived danger of the current situation. We demonstrate improvements in several cases.

Abstract Crowd simulators are commonly used to populate movie or game scenes in the entertainment industry. Even though it is crucial to consider the presence of groups for the believability of a virtual crowd, most crowd simulations only take into account individual characters or a limited set of group behaviors. We introduce a unified solution that allows for simulations of crowds that have diverse group properties such as social groups, marches, tourists and guides, etc. We extend the Velocity Obstacle approach for agent‐based crowd simulations by introducing Velocity Connection; the set of velocities that keep agents moving together while avoiding collisions and achieving goals. We demonstrate our approach to be robust, controllable, and able to cover a large set of group behaviors.

A navigation mesh is a representation of a 2D or 3D virtual environment that enables path planning and crowd simulation for walking characters. Various state-of-the-art navigation meshes exist, but there is no standardized way of evaluating or comparing them. Each implementation is in a different state of maturity, has been tested on different hardware, uses different example environments, and may have been designed with a different application in mind.

We present an interactive motion planning algorithm to compute plausible trajectories for high-DOF human-like characters. Given a discrete sequence of contact configurations, we use a three-phase optimization approach to ensure that the resulting trajectory is collision-free, smooth, and satisfies dynamic balancing constraints. Our approach can directly compute dynamically balanced and natural-looking motions at interactive frame rates and is considerably faster than prior methods. We highlight its performance on complex human motion benchmarks corresponding to walking, climbing, crawling, and crouching, where the discrete configurations are generated from a kinematic planner or extracted from motion capture datasets.

Abstract Character animation based on motion capture provides intrinsically plausible results, but lacks the flexibility of procedural methods. Motion editing methods partially address this limitation by adapting the animation to small deformations of the environment. We extend one such method, the so‐called relationship descriptors, to tackle the issue of motion editing under large environment deformations. Large deformations often result in joint limits violation, loss of balance, or collisions. Our method handles these situations by automatically detecting and re‐positioning invalidated contacts. The new contact configurations are chosen to preserve the mechanical properties of the original contacts in order to provide plausible support phases. When it is not possible to find an equivalent contact, a procedural animation is generated and blended with the original motion. Thanks to an optimization scheme, the resulting motions are continuous and preserve the style of the reference motions. The method is fully interactive and enables the motion to be adapted on‐line even in case of large changes of the environment. We demonstrate our method on several challenging scenarios, proving its immediate application to 3D animation softwares and video games.

A typical crowd engine pipeline animates numerous moving characters according to a two-step process: global trajectories are generated by a crowd simulator, whereas full body motions are generated by animation engines. Because interactions are only considered at the first stage, animations sometimes lead to residual collisions and/or characters walking as if they were alone, showing no sign to the influence of others. In this paper, we investigate the value of adding shoulder motions to characters passing at close distances on the perceived visual quality of crowd animations (i.e., perceived residual collisions and animation naturalness). We present two successive perceptual experiments exploring this question where we investigate first, local interactions between two isolated characters, and second, crowd scenarios. The first experiment shows that shoulder motions have a strong positive effect on both perceived residual collisions and animation naturalness. The second experiment demonstrates that the effect of shoulder motions on animation naturalness is preserved in the context of crowd scenarios, even though the complexity of the scene is largely increased. Our general conclusion is that adding secondary motions in character interactions has a significant impact on the visual quality of crowd animations, with a very light impact on the computational cost of the whole animation pipeline. Our results advance crowd animation techniques by enhancing the simulation of complex interactions between crowd characters with simple secondary motion triggering techniques.

Microscopic crowd simulators rely on models of local interaction (e.g. collision avoidance) to synthesize the individual motion of each virtual agent. The quality of the resulting motions heavily depends on this component, which has significantly improved in the past few years. Recent advances have been in particular due to the introduction of a short-horizon motion prediction strategy that enables anticipated motion adaptation during local interactions among agents. However, the simplicity of prediction techniques of existing models somewhat limits their domain of validity. In this paper, our key objective is to significantly improve the quality of simulations by expanding the applicable range of motion predictions. To this end, we present a novel local interaction algorithm with a new context-aware, probabilistic motion prediction model. By context-aware, we mean that this approach allows crowd simulators to account for many factors, such as the influence of environment layouts or in-progress interactions among agents, and has the ability to simultaneously maintain several possible alternate scenarios for future motions and to cope with uncertainties on sensing and other agent's motions. Technically, this model introduces "collision probability fields" between agents, efficiently computed through the cumulative application of Warp Operators on a source Intrinsic Field. We demonstrate how this model significantly improves the quality of simulated motions in challenging scenarios, such as dense crowds and complex environments.

Artists, animation and game designers are in demand for solutions to easily populate large virtual environments with crowds that satisfy desired visual features. This paper presents a method to intuitively populate virtual environments by specifying two key features: localized density, being the amount of agents per unit of surface, and localized flow, being the direction in which agents move through a unit of surface. The technique we propose is also time-independant, meaning that whatever the time in the animation, the resulting crowd satisfies both features. To achieve this, our approach relies on the Crowd Patches model. After discretizing the environment into regular patches and creating a graph that links these patches, an iterative optimization process computes the local changes to apply on each patch (increasing/reducing the number of agents in each patch, updating the directions of agents in the patch) in order to satisfy overall density and flow constraints. A specific stage is then introduced after each iteration to avoid the creation of local loops by using a global pathfinding process. As a result, the method has the capacity of generating large realistic crowds in minutes that endlessly satisfy both user specified densities and flow directions, and is robust to contradictory inputs. At last, to ease the design the method is implemented in an artist-driven tool through a painting interface.

In this paper, we investigate obstacle avoidance behavior during real walking in a large immersive projection setup. We analyze the walking behavior of users when avoiding real and virtual static obstacles. In order to generalize our study, we consider both anthropomorphic and inanimate objects, each having his virtual and real counterpart. The results showed that users exhibit different locomotion behaviors in the presence of real and virtual obstacles, and in the presence of anthropomorphic and inanimate objects. Precisely, the results showed a decrease of walking speed as well as an increase of the clearance distance (i. e., the minimal distance between the walker and the obstacle) when facing virtual obstacles compared to real ones. Moreover, our results suggest that users act differently due to their perception of the obstacle: users keep more distance when the obstacle is anthropomorphic compared to an inanimate object and when the orientation of anthropomorphic obstacle is from the profile compared to a front position. We discuss implications on future large shared immersive projection spaces.

Abstract Representing the majority of living animals, insects are the most ubiquitous biological organisms on Earth. Being able to simulate insect swarms could enhance visual realism of various graphical applications. However, the very complex nature of insect behaviors makes its simulation a challenging computational problem. To address this, we present a general biologically‐inspired framework for visual simulation of insect swarms. Our approach is inspired by the observation that insects exhibit emergent behaviors at various scales in nature. At the low level, our framework automatically selects and configures the most suitable steering algorithm for the local collision avoidance task. At the intermediate level, it processes insect trajectories into piecewise‐linear segments and constructs probability distribution functions for sampling waypoints. These waypoints are then evaluated by the Metropolis‐Hastings algorithm to preserve global structures of insect swarms at the high level. With this biologically inspired, data‐driven approach, we are able to simulate insect behaviors at different scales and we evaluate our simulation using both qualitative and quantitative metrics. Furthermore, as insect data could be difficult to acquire, our framework can be adopted as a computer‐assisted animation tool to interpret sketch‐like input as user control and generate simulations of complex insect swarming phenomena.

In this paper, we propose a novel idea to address the problem of fast computation of stable force-closure grasp configurations for a multifingered hand and a 3-D rigid object represented as a polygonal soup model. The proposed method performs a low-level shape exploration by wrapping multiple cords around the object in order to quickly isolate promising grasping regions. Around these regions, we compute grasp configurations by applying a variant of the close-until-contact procedure to find the contact points. The finger kinematics and the contact information are then used to filter out unstable grasps. Through many simulated examples with three different anthropomorphic hands, we demonstrate that, compared with previous grasp planners such as the generic grasp planner in Simox, the proposed grasp planner can synthesize grasps that are more natural-looking for humans (as measured by the grasp quality measure skewness) for objects with complex geometries in a short amount of time. Unlike many other planners, this is achieved without costly model preprocessing such as segmentation by parts and medial axis extraction.

When avoiding a group, a walker has two possibilities: either he goes through it or around it. Going through very dense groups or around huge ones would not seem natural and could break any sense of presence in a virtual environment. This paper aims to enable crowd simulators to handle such situations correctly. To this end, we need to understand how real humans decide to go through or around groups. As a first hypothesis, we apply the Principle of Minimum Energy (PME) on different group sizes and density. According to this principle, a walker should go around small and dense groups whereas he should go through large and sparse groups. Such principle has already been used for crowd simulation; the novelty here is to apply it to decide on a global avoidance strategy instead of local adaptations only. Our study quantifies decision thresholds. However, PME leaves some inconclusive situations for which the two solutions paths have similar energetic costs. In a second part, we propose an experiment to corroborate PME decisions thresholds with real observations. As controlling the factors of an experiment with many people is extremely hard, we propose to use Virtual Reality as a new method to observe human behavior. This work represents the first crowd simulation algorithm component directly designed from a VR-based study. We also consider the role of secondary factors in inconclusive situations. We show the influence of the group appearance and direction of relative motion in the decision process. Finally, we draw some guidelines to integrate our conclusions to existing crowd simulators and show an example of such integration. We evaluate the achieved improvements.

In this paper, we present a new model to simulate following behavior. This model is based on a dynamic following distance that changes according to the follower's speed and to the leader's motion. The following distance is associated with a prediction of the leader's future position to give a following ideal position. We show the resulting following trajectory and detail the importance of the distance variation in different situations. The model is evaluated using real data. We demonstrate the capacity of our model to reproduce macroscopic patterns and show that it is also able to synthesize trajectories similar to real ones. Finally, we compare our results with other following models and point out the improvements.

Over the past few years, simulating crowds in virtual environments has become an important tool to give life to virtual scenes; be it movies, games, training applications, etc. An important part of crowd simulation is the way that people move from one place to another. This paper concentrates on improving the crowd patches approach proposed by Yersin et al. [Yersin et al. 2009] that aims on efficiently animating ambient crowds in a scene. This method is based on the construction of animation blocks (called patches) concatenated together under some constraints to create larger and richer animations with limited run-time cost. Specifically, an optimization based approach to generate smooth collision free trajectories for crowd patches is proposed. The contributions of this work to the crowd patches framework are threefold; firstly a method to match the end points of trajectories based on the Gale-Shapley algorithm [Gale and Shapley 1962] is proposed that takes into account preferred velocities and space coverage, secondly an improved algorithm for collision avoidance is proposed that gives natural appearance to trajectories and finally a cubic spline approach is used to smooth out generated trajectories. We demonstrate several examples of patches and how they were improved by the proposed method, some limitations and directions for future improvements.

Abstract We introduce “Crowd Sculpting”: a method to interactively design populated environments by using intuitive deformation gestures to drive both the spatial coverage and the temporal sequencing of a crowd motion. Our approach assembles large environments from sets of spatial elements which contain inter‐connectible, periodic crowd animations. Such a “Crowd Patches” approach allows us to avoid expensive and difficult‐to‐control simulations. It also overcomes the limitations of motion editing, that would result into animations delimited in space and time. Our novel methods allows the user to control the crowd patches layout in ways inspired by elastic shape sculpting: the user creates and tunes the desired populated environment through stretching, bending, cutting and merging gestures, applied either in space or time. Our examples demonstrate that our method allows the space‐time editing of very large populations and results into endless animation, while offering real‐time, intuitive control and maintaining animation quality.

Abstract We present a novel framework to evaluate multi‐agent crowd simulation algorithms based on real‐world observations of crowd movements. A key aspect of our approach is to enable fair comparisons by automatically estimating the parameters that enable the simulation algorithms to best fit the given data. We formulate parameter estimation as an optimization problem, and propose a general framework to solve the combinatorial optimization problem for all parameterized crowd simulation algorithms. Our framework supports a variety of metrics to compare reference data and simulation outputs. The reference data may correspond to recorded trajectories, macroscopic parameters, or artist‐driven sketches. We demonstrate the benefits of our framework for example‐based simulation, modeling of cultural variations, artist‐driven crowd animation, and relative comparison of some widely‐used multi‐agent simulation algorithms.

This paper is about interactions between real and virtual humans. We are interested in whole body and emotionally tinted situations of interactions. We focus on the daily situation of walking together. We propose two experiments. In a first experiment, we measure the effect of emotions on the kinematics and metrics of interactions between two walkers. Then, in a second experiment, we reproduce a similar situation of interaction between a real subject and a virtual walker expressing emotions. We perform comparisons between real-real interactions and real-virtual ones. We show promising results: similar effects are observed on the kinematics of interactions in both experiments. We implicitly demonstrate the ability of real subjects to perceive emotions expressed by a virtual character through whole body motion. We show that real subjects' reaction to the behavior of an expressive virtual character complies with their reaction to the behavior of an expressive real human walker. This result is one step toward the use of such virtual reality platform to study social interactions through fully controlled experiments.

In this paper, we propose a novel idea to address the problem of fast computation of enveloping grasp configurations for a multi-fingered hand with 3D polygonal models represented as polygon soups. The proposed method performs a low-level shape matching by wrapping multiple cords around an object in order to quickly isolate promising grasping spots. From these spots, hand palm posture can be computed followed by a standard close-until-contact procedure to find the contact points. Along with the contacts information, the finger kinematics is then used to filter the unstable grasps. Through multiple simulated examples with a twelve degrees-of-freedom anthropomorphic hand, we demonstrate that our method can compute good grasps for objects with complex geometries in a short amount of time. Best of all, this is achieved without complex model preprocessing like segmentation by parts and medial axis extraction.

Virtual walking, a fundamental task in Virtual Reality (VR), is greatly influenced by the locomotion interface being used, by the specificities of input and output devices, and by the way the virtual environment is represented. No matter how virtual walking is controlled, the generation of realistic virtual trajectories is absolutely required for some applications, especially those dedicated to the study of walking behaviors in VR, navigation through virtual places for architecture, rehabilitation and training. Previous studies focused on evaluating the realism of locomotion trajectories have mostly considered the result of the locomotion task (efficiency, accuracy) and its subjective perception (presence, cybersickness). Few focused on the locomotion trajectory itself, but in situation of geometrically constrained task. In this paper, we study the realism of unconstrained trajectories produced during virtual walking by addressing the following question: did the user reach his destination by virtually walking along a trajectory he would have followed in similar real conditions? To this end, we propose a comprehensive evaluation framework consisting on a set of trajectographical criteria and a locomotion model to generate reference trajectories. We consider a simple locomotion task where users walk between two oriented points in space. The travel path is analyzed both geometrically and temporally in comparison to simulated reference trajectories. In addition, we demonstrate the framework over a user study which considered an initial set of common and frequent virtual walking conditions, namely different input devices, output display devices, control laws, and visualization modalities. The study provides insight into the relative contributions of each condition to the overall realism of the resulting virtual trajectories.

Abstract While walking through a crowd, a pedestrian experiences a large number of interactions with his neighbors. The nature of these interactions is varied, and it has been observed that macroscopic phenomena emerge from the combination of these local interactions. Crowd models have hitherto considered collision avoidance as the unique type of interactions between individuals, few have considered walking in groups. By contrast, our paper focuses on interactions due to the following behaviors of pedestrians. Following is frequently observed when people walk in corridors or when they queue. Typical macroscopic stop‐and‐go waves emerge under such traffic conditions. Our contributions are, first, an experimental study on following behaviors, second, a numerical model for simulating such interactions, and third, its calibration, evaluation and applications. Through an experimental approach, we elaborate and calibrate a model from microscopic analysis of real kinematics data collected during experiments. We carefully evaluate our model both at the microscopic and the macroscopic levels. We also demonstrate our approach on applications where following interactions are prominent.

In human crowds as well as in many animal societies, local interactions among individuals often give rise to self-organized collective organizations that offer functional benefits to the group. For instance, flows of pedestrians moving in opposite directions spontaneously segregate into lanes of uniform walking directions. This phenomenon is often referred to as a smart collective pattern, as it increases the traffic efficiency with no need of external control. However, the functional benefits of this emergent organization have never been experimentally measured, and the underlying behavioral mechanisms are poorly understood. In this work, we have studied this phenomenon under controlled laboratory conditions. We found that the traffic segregation exhibits structural instabilities characterized by the alternation of organized and disorganized states, where the lifetime of well-organized clusters of pedestrians follow a stretched exponential relaxation process. Further analysis show that the inter-pedestrian variability of comfortable walking speeds is a key variable at the origin of the observed traffic perturbations. We show that the collective benefit of the emerging pattern is maximized when all pedestrians walk at the average speed of the group. In practice, however, local interactions between slow- and fast-walking pedestrians trigger global breakdowns of organization, which reduce the collective and the individual payoff provided by the traffic segregation. This work is a step ahead toward the understanding of traffic self-organization in crowds, which turns out to be modulated by complex behavioral mechanisms that do not always maximize the group's benefits. The quantitative understanding of crowd behaviors opens the way for designing bottom-up management strategies bound to promote the emergence of efficient collective behaviors in crowds.

Simulating the local behavior of small pedestrian groups among a crowd is an emerging problem. Small groups are commonly found in general crowds and their behavior are significantly distinguished from that of individual pedestrians. In this paper, we propose a novel approach to simulate the walking behavior of such small groups. We construct dynamic group formations which not only facilitate easy communication between members within the same group but also adapt to the environment constraint. Moreover, an extended collision avoidance algorithm based on Optimal Reciprocal Collision Avoidance (ORCA) is proposed taking into account the factors of group cohesion and reaction flexibility so as to generate diverse interaction behavior between walkers. Several examples demonstrate the efficiency of the proposed algorithm.

Annotating unlabeled motion captures plays an important role in Computer Animation for motion analysis and motion edition purposes. Locomotion is a difficult case study as all the limbs of the human body are involved whereas a low-dimensional global motion is performed. The oscillatory nature of the locomotion makes difficult the distinction between straight steps and turning ones, especially for subtle orientation changes. In this paper we propose to geometrically model the center of mass trajectory during locomotion as a C-continuous circular arcs sequence. Our model accurately analyzes the global motion into the velocity-curvature space. An experimental study demonstrates that an invariant law links curvature and velocity during straight walk. We finally illustrate how the resulting law can be used for annotation purposes: any unlabeled motion captured walk can be transformed into an annotated sequence of straight and turning steps. Several examples demonstrate the robustness of our approach and give comparison with classical threshold-based techniques. Copyright © 2011 John Wiley & Sons, Ltd.

ABSTRACT The seamless integration of virtual characters into dynamic scenes captured by video is a challenging problem. In order to achieve consistent composite results, both the virtual and real characters must share the same geometrical constraints and their interactions must follow some common sense. One essential question is how to detect the motion of real objects—such as real characters moving in the video—and how to steer virtual characters accordingly to avoid unrealistic collisions. We propose an online solution. First, by analysis of the input video, the motion states of the real pedestrians are recovered into a common world 3D coordinate system. Meanwhile, a simplified accuracy measurement is defined to represent the confidence of the motion estimate. Then, under the constraints imposed by the real dynamic objects, the motion of virtual characters are accommodated by a uniform steering model. The final step is to merge virtual objects back to the real video scene by taking into account visibility and occlusion constraints between real foreground objects and virtual ones. Several examples demonstrate the efficiency of the proposed algorithm. Copyright © 2011 John Wiley & Sons, Ltd.

The performance of an interactive virtual crowd system for entertainment purposes can be greatly improved by setting a level-of-details (LOD) strategy: in distant areas, collision avoidance can even be stealthy disabled to drastically speed-up simulation and to handle huge crowds. The greatest difficulty is then to select LODs to progressively simplify simulation in an imperceptible but efficient manner. The main objective of this work is to experimentally evaluate spectators' ability to detect the presence of collisions in simulations. Factors related to the conditions of observation and simulation are studied, such as the camera angles, distance to camera, level of interpenetration or crowd density. Our main contribution is to provide a LOD selection function resulting from two perceptual studies allowing crowd system designers to scale a simulation by relaxing the collision avoidance constraint in a least perceptible manner. The relaxation of this constraint is an important source for computational resources savings. Our results reveal several misconceptions in previously used LOD selection functions and suggest yet unexplored variables to be considered. We demonstrate our function efficiency over several evaluation scenarios.

In the everyday exercise of controlling their locomotion, humans rely on their optic flow of the perceived environment to achieve collision-free navigation. In crowds, in spite of the complexity of the environment made of numerous obstacles, humans demonstrate remarkable capacities in avoiding collisions. Cognitive science work on human locomotion states that relatively succinct information is extracted from the optic flow to achieve safe locomotion. In this paper, we explore a novel vision-based approach of collision avoidance between walkers that fits the requirements of interactive crowd simulation. By simulating humans based on cognitive science results, we detect future collisions as well as the level of danger from visual stimuli. The motor-response is twofold: a reorientation strategy prevents future collision, whereas a deceleration strategy prevents imminent collisions. Several examples of our simulation results show that the emergence of self-organized patterns of walkers is reinforced using our approach. The emergent phenomena are visually appealing. More importantly, they improve the overall efficiency of the walkers' traffic and avoid improbable locking situations.

Populating virtual environments (VEs) with large crowds is a subject that has been tackled for several years. Solutions have been proposed to offer realistic trajectories as well as interactivity, but limitations remain on the environment dimensions with respect to population density. In this paper, we extend the concept of motion patches [Lee et al. 2006] to densely populate large environments. We build a population from a set of blocks containing a pre-computed local crowd simulation. Each block is called a crowd patch. We address the problem of computing patches, assembling them to create VEs, and controlling their content to answer designers' needs. Our major contribution is to provide a drastic lowering of computation needs for simulating a virtual crowd at run-time. We can thus handle dense populations in large-scale environments with performances never reached so far. Our results illustrate the real-time population of a potentially infinite city with realistic and varied crowds interacting with each other and their environment. We discuss the advantages and drawbacks of the proposed solution, and its possible improvements in the future.

This paper presents a solution to interactive navigation planning and real-time simulation of a very large number of entities moving in a virtual environment. From the environment geometry analysis, we deduce a structure called navigation graph, which is the base to our method. After the description of this structure, we introduce a set of algorithms dedicated to answer navigation queries with a set of various solution paths and to execute the planned navigation in an efficient manner. We equally demonstrate method performance and robustness over several examples.

Abstract This paper addresses the problem of virtual pedestrian autonomous navigation for crowd simulation. It describes a method for solving interactions between pedestrians and avoiding inter‐collisions. Our approach is agent‐based and predictive: each agent perceives surrounding agents and extrapolates their trajectory in order to react to potential collisions. We aim at obtaining realistic results, thus the proposed model is calibrated from experimental motion capture data. Our method is shown to be valid and solves major drawbacks compared to previous approaches such as oscillations due to a lack of anticipation. We first describe the mathematical representation used in our model, we then detail its implementation, and finally, its calibration and validation from real data.

Abstract This paper introduces a framework for real‐time simulation and rendering of crowds navigating in a virtual environment. The solution first consists in a specific environment preprocessing technique giving rise to navigation graphs, which are then used by the navigation and simulation tasks. Second, navigation planning interactively provides various solutions to the user queries, allowing to spread a crowd by individualizing trajectories. A scalable simulation model enables the management of large crowds, while saving computation time for rendering tasks. Pedestrian graphical models are divided into three rendering fidelities ranging from billboards to dynamic meshes, allowing close‐up views of detailed digital actors with a large variety of locomotion animations. Examples illustrate our method in several environments with crowds of up to 35 000 pedestrians with real‐time performance. Copyright © 2006 John Wiley & Sons, Ltd.

Abstract Virtual mannequins need to navigate in order to interact with their environment. Their autonomy to accomplish navigation tasks is ensured by locomotion controllers. Control inputs can be user‐defined or automatically computed to achieve high‐level operations (e.g. obstacle avoidance). This paper presents a locomotion controller based on a motion capture edition technique. Controller inputs are the instantaneous linear and angular velocities of the walk. Our solution works in real time and supports at any time continuous changes of inputs. The controller combines three main components to synthesize locomotion animations in a four‐stage process. First, the Motion Library stores motion capture samples. Motion captures are analysed to compute quantitative characteristics. Second, these characteristics are represented in a linear control space. This geometric representation is appropriate for selecting and weighting three motion samples with respect to the input state. Third, locomotion cycles are synthesized by blending the selected motion samples. Blending is done in the frequency domain. Lastly, successive postures are extracted from the synthesized cycles in order to complete the animation of the moving mannequin. The method is demonstrated in this paper in a locomotion‐planning context. Copyright © 2006 John Wiley & Sons, Ltd.

This paper presents an approach to automatically compute animations for virtual (human-like and robot) characters cooperating to move bulky objects in cluttered environments. The main challenge is to deal with 3D collision avoidance while preserving the believability of the agent's behaviors. To accomplish the coordinated task, a geometric and kinematic decoupling of the system is proposed. This decomposition enables us to plan a collision-free path for a reduced system, then to animate locomotion and grasping behaviors independently, and finally to automatically tune the animation to avoid residual collisions. These three steps are applied consecutively to synthesize an animation. The different techniques used, such as probabilistic path planning, locomotion controllers, inverse kinematics and path planning for closed kinematic chains are explained, and the way to integrate them into a single scheme is described.

This paper presents some evolutions over the locomotion planning problem for digital actors. The solution is based both on probabilistic motion planning and on motion capture blending and warping. The paper particularly focuses on a new collision avoidance technique: while the legs and the pelvis of the digital actor follow a planned path, the animation of the upper part of the body is updated for 3D collision avoidance purposes.

This paper presents a method for animating human characters, especially dedicated to walk planning problems. The method is integrated in a randomized motion planning scheme, including a steering method dedicated to human walk. This steering method integrates a character motion controller assuming realistic animations. The navigation of the character through a virtual environment is modeled as a composition of Bezier curves. The controller is based on motion capture data editing techniques. This approach satisfies some essential computer graphics criteria: a realistic result, a low response time, a collision-free motion in possibly constrained 3D environments. The approach has been implemented and successfully demonstrated on several examples.