This paper describes the process of design, analysis, and scoring with both subjective and objective criteria, of a set of possible haptic inceptors configurations to be used as primary flight control devices for a class of future tilt rotor vehicles. Several scoring criteria were adopted like ergonomics, cognitive compatibility, easiness of recovery in fault situations, and expected workload among several others. Each criterion was weighted to obtain a scoring matrix for a total of 18 different configurations, some "classic", some novel, leading to an overall ranking. The matrix was used to select the best candidate configurations to be tested using a human-in-the-loop simulator in the context of the EU Funded CleanSky2 SMARTeR Project. After a final review phase, the best solution will be adopted to actually fly the Leonardo NextGen Civil Tilt Rotor Technological Demonstrator in the context of the EU Funded CleanSky2 SAIS Project.

Virtual Reality supports collaboration among partners across departments and fields, independent of physical boundaries. Virtual reality applications can solve the time and cost consuming logistic problem that companies encounter when sending experts to remote locations. However, it is not yet clear how effective partners collaborate when in remote locations. In one experiment, we examined whether partners who are physically in the same room and interact with each other before they start collaborating affects performance compared to collaborators who meet and interact only within the virtual space. Participants had to solve a Rubik's cube type three-dimensional puzzle by arranging cubes that varied in color within a solution space in such a way, so that each side of the solution space showed a single color. Participants were immersed within a virtual environment and in one condition participants were collocated in the same room (local condition), while in the other one they were located in different rooms (remote condition). Results showed that collaborators in both conditions successfully completed the task but performance was better during the local compared to the remote condition.

The human controller (HC) in manual control of a dynamical system often follows a visible and predictable reference path (target). The HC can adopt a control strategy combining closed-loop feedback and an open-loop feedforward response. The effects of the target signal waveform shape and the system dynamics on the human feedforward dynamics are still largely unknown, even for common, stable, vehicle-like dynamics. This paper studies the feedforward dynamics through computer model simulations and compares these to system identification results from human-in-the-loop experimental data. Two target waveform shapes are considered, constant velocity ramp segments and constant acceleration parabola segments. Furthermore, three representative vehicle-like system dynamics are considered: 1) a single integrator (SI); 2) a second-order system; and 3) a double integrator. The analyses show that the HC utilizes a combined feedforward/feedback control strategy for all dynamics with the parabola target, and for the SI and second-order system with the ramp target. The feedforward model parameters are, however, very different between the two target waveform shapes, illustrating the adaptability of the HC to task variables. Moreover, strong evidence of anticipatory control behavior in the HC is found for the parabola target signal. The HC anticipates the future course of the parabola target signal given extensive practice, reflected by negative feedforward time delay estimates.

Creating metrically accurate avatars is important for many applications such as virtual clothing try-on, ergonomics, medicine, immersive social media, telepresence, and gaming. Creating avatars that precisely represent a particular individual is challenging however, due to the need for expensive 3D scanners, privacy issues with photographs or videos, and difficulty in making accurate tailoring measurements. We overcome these challenges by creating "The Virtual Caliper", which uses VR game controllers to make simple measurements. First, we establish what body measurements users can reliably make on their own body. We find several distance measurements to be good candidates and then verify that these are linearly related to 3D body shape as represented by the SMPL body model. The Virtual Caliper enables novice users to accurately measure themselves and create an avatar with their own body shape. We evaluate the metric accuracy relative to ground truth 3D body scan data, compare the method quantitatively to other avatar creation tools, and perform extensive perceptual studies. We also provide a software application to the community that enables novices to rapidly create avatars in fewer than five minutes. Not only is our approach more rapid than existing methods, it exports a metrically accurate 3D avatar model that is rigged and skinned.

Touchscreens are a common fixture in current vehicles. With autonomous driving, we can expect touch interaction with such in-vehicle media systems to exponentially increase. In spite of vehicle suspension systems, road perturbations will continue to exert forces that can render in-vehicle touch interaction challenging. Using a motion simulator, we investigate how different vehicle speeds interact with road features (i.e., speed bumps) to influence touch interaction. We determine their effect on pointing accuracy and task completion time. We show that road bumps have a significant effect on touch input and can decrease accuracy by 19%. In light of this, we developed a Random Forest (RF) model that improves touch accuracy by 32.0% on our test set and by 22.5% on our validation set. As the lightweight model uses only features that can easily be determined through inertial measurement units, this model could be easily deployed in current automobiles.

Take-over requests (TORs) in highly automated vehicles are cues that prompt users to resume control. TORs however, are often evaluated in non-moving driving simulators. This ignores the role of motion, an important source of information for users who have their eyes off the road while engaged in non-driving related tasks. We ran a user study in a moving-base driving simulator to investigate the effect of motion on TOR responses. We found that with motion, user responses to TORs vary depending on the road context where TORs are issued. While previous work showed that participants are fast to respond to urgent cues, we show that this is true only when TORs are presented on straight roads. Urgent cues issued on curved roads elicit slower responses than non-urgent cues on curved roads. Our findings indicate that TORs should be designed to be aware of road context to accommodate natural user responses.

Design recommendations for notifications are typically based on user performance and subjective feedback. In comparison, there has been surprisingly little research on how designed notifications might be processed by the brain for the information they convey. The current study uses EEG/ERP methods to evaluate auditory notifications that were designed to cue long-distance truck drivers for task-management and driving conditions, particularly for automated driving scenarios. Two experiments separately evaluated naive students and professional truck drivers for their behavioral and brain responses to auditory notifications, which were either auditory icons or verbal commands. Our EEG/ERP results suggest that verbal commands were more readily recognized by the brain as relevant targets, but that auditory icons were more likely to update contextual working memory. Both classes of notifications did not differ on behavioral measures. This suggests that auditory icons ought to be employed for communicating contextual information and verbal commands, for urgent requests.

Indoor operation of unmanned aerial vehicles (UAV s) poses many challenges due to the lack of GPS signal and cramped spaces. The presence of obstacles in an unfamiliar environment requires reliable state estimation and active algorithms to prevent collisions. In this paper, we present a teleoperated quadrotor UAV platform equipped with an onboard miniature computer and a minimal set of sensors for this task. The platform is capable of highly accurate state-estimation, tracking of desired velocity commanded by the user and ensuring collision-free navigation. The robot estimates its linear velocity through a Kalman filter integration of inertial and optical flow (OF) readings with corresponding distance measurements. An RGB-D camera serves the purpose of providing visual feedback to the operator and depth measurements to build a probabilistic, robo-centric obstacle model, allowing the robot to avoid collisions. The platform is thoroughly validated in experiments in an obstacle rich environment.

This paper presents the experimental evaluation of a novel haptic shared control system. The proposed system was designed to help human operators to perform a maneuver that is not known in advance. A Pilot Intent Estimator (PIE) was implemented to estimate the unknown trajectory that a human pilot intends to follow. Then, a haptic feedback was designed to mimic the behavior of a skilled pilot that tracks the estimated trajectory. A human-in-the-loop experiment was performed to evaluate the developed shared control system in a 2 Degrees-of- Freedom (DoF) control task. Results showed the effectiveness of the PIE to estimate the correct pilot intended path. Furthermore, the developed haptic system helped participants to achieve better performance to follow the estimated trajectory, compared to manual control.

Realistic manual control tasks typically involve predictable target signals and random disturbances. The human controller (HC) is hypothesized to use a feedforward control strategy for target-following, in addition to feedback control for disturbance-rejection. Little is known about human feedforward control, partly because common system identification methods have difficulty in identifying whether, and (if so) how, the HC applies a feedforward strategy. In this paper, an identification procedure is presented that aims at an objective model selection for identifying the human feedforward response, using linear time-invariant autoregressive with exogenous input models. A new model selection criterion is proposed to decide on the model order (number of parameters) and the presence of feedforward in addition to feedback. For a range of typical control tasks, it is shown by means of Monte Carlo computer simulations that the classical Bayesian information criterion (BIC) leads to selecting models that contain a feedforward path from data generated by a pure feedback model: "false-positive" feedforward detection. To eliminate these false-positives, the modified BIC includes an additional penalty on model complexity. The appropriate weighting is found through computer simulations with a hypothesized HC model prior to performing a tracking experiment. Experimental human-in-the-loop data will be considered in future work. With appropriate weighting, the method correctly identifies the HC dynamics in a wide range of control tasks, without false-positive results.

Motion cueing algorithms are used in motion simulation to map the inertial vehicle motion onto the limited simulator motion space. This mapping causes mismatches between the unrestricted visual motion and the constrained inertial motion, which results in perceived motion incongruence (PMI). It is still largely unknown what exactly causes visual and inertial motion in a simulator to be perceived as incongruent. Current methods for measuring motion incongruence during motion simulation result in time-invariant measures of the overall incongruence, which makes it difficult to determine the relevance of the individual and short-duration mismatches between visual and inertial motion cues. In this paper, a novel method is presented to subjectively measure the time-varying PMI continuously throughout a simulation. The method is analyzed for reliability and validity of its measurements, as well as for its applicability in relating physical short-duration cueing errors to PMI. The analysis shows that the method is reliable and that the results can be used to obtain a deeper insight into the formation of motion incongruence during driving simulation.

In this paper, we propose and experimentally verify a distributed formation control algorithm for a group of multirotor unmanned aerial vehicles (UAVs). The algorithm brings the whole group of UAVs simultaneously to a prescribed submanifold that determines the formation shape in an asymptotically stable fashion in two- and three-dimensional environments. The complete distributed control framework is implemented with the combination of a fast model predictive control method executed at 50 Hz on low-power computers onboard multirotor UAVs and validated via a series of hardware-in-the-loop simulations and real-robot experiments. The experiments are configured to study the control performance in various formation cases of arbitrary time-varying (e.g., expanding, shrinking, or moving) shapes. In the actual experiments, up to four multirotors have been implemented to form arbitrary triangular, rectangular, and circular shapes drawn by the operator via a human-robot interaction device. We also carry out hardware-in-the-loop simulations using up to six onboard computers to achieve spherical formations and a formation moving through obstacles.

Physical human-UAV interaction (PHUI) with Unmanned Aerial Vehicles (UAV) has many possible applications, and a few recent works [1], [2] have shown that it is possible to perform PHUI with a standard quadrotor. However, the intrinsic underactuation of quadrotors may hinder the interaction task and also cause safety issues. In this paper, we present an admittance control-based scheme to perform PHUI with a fully actuated UAV. The system also benefits from the robustness provided by our fully actuated UAV implementation of the adaptive super twisting controller (ASTC). While validating our system in simulation, we also show the superior performance of the fully actuated UAV with respect to a standard quadcopter.

This paper investigates the use of a variable haptic feedback for training a disturbance rejection task. The haptic feedback was designed as a Force-Stiffness feedback. Throughout the training, Force and Stiffness feedback are decreased to progressively give more control authority to the human operator. The training method was tested in a human-in-the-loop experiment. In the experiment, participants were split into three groups: variable haptic aid (VHA), constant haptic aid (CHA) and no haptic aid (NoHA). The VHA and CHA groups performed a first training phase with variable and constant haptic feedback respectively, followed by an evaluation phase without external aids. The NoHA group performed the entire experiment without external aids. Results showed that in the training phase both VHA and CHA groups performed better than NoHA group. In the evaluation phase though, only the VHA group obtained better performances than the NoHA group. Specifically, participants were able to quickly recover similar performances to those obtained at the end of the training phase. Thus, the variable haptic training proved to be more effective than the constant haptic training and manual control at helping participants learn the task.

Biodynamic feedthrough (BDFT) refers to the feedthrough of vehicle accelerations through the human body, leading to involuntary control device inputs. BDFT impairs control performance in a large range of vehicles under various circumstances. Research shows that BDFT strongly depends on adaptations in the neuromuscular admittance dynamics of the human body. This paper proposes a model-based approach of BDFT mitigation that accounts for these neuromuscular adaptations. The method was tested, as proof-of-concept, in an experiment where participants inside a motion simulator controlled a simulated vehicle through a virtual tunnel. Through evaluating tracking performance and control effort with and without motion disturbance active and with and without cancellation active, the effectiveness of the cancellation was evaluated. Results show that the cancellation approach is successful: the detrimental effects of BDFT were largely removed.

In this paper, we present a collision-free indoor navigation algorithm for teleoperated multirotor Unmanned Aerial Vehicles (UAVs). Assuming an obstacle rich environment, the algorithm keeps track of detected obstacles in the local surroundings of the robot. The detection part of the algorithm is based on measurements from an RGB-D camera and a Bin-Occupancy filter capable of tracking an unspecified number of targets. We use the estimate of the robot's velocity to update the obstacles state when they leave the direct field of view of the sensor. The avoidance part of the algorithm is based on the Model Predictive Control approach. By predicting the possible future obstacles states, it filters the operator commands to prevent collisions. The method is validated on a platform equipped with its own computational unit, which makes it self-sufficient in terms of external CPUs.

In this paper we present a robust quadrotor controller for tracking a reference trajectory in presence of uncertainties and disturbances. A Super Twisting controller is implemented using the recently proposed gain adaptation law [1], [2], which has the advantage of not requiring the knowledge of the upper bound of the lumped uncertainties. The controller design is based on the regular form of the quadrotor dynamics, without separation in two nested control loops for position and attitude. The controller is further extended by a feedforward dynamic inversion control that reduces the effort of the sliding mode controller. The higher order quadrotor dynamic model and proposed controller are validated using a SimMechanics physical simulation with initial error, parameter uncertainties, noisy measurements and external perturbations.

This paper addresses the problem of cooperative aerial transportation of an object using a team of quadrotors. The approach presented to solve this problem accounts for the full dynamics of the system and it is inspired by the literature on reconfigurable cable-driven parallel robots (RCDPR). Using the modelling convention of RCDPR it is derived a direct relation between the motion of the quadrotors and the motion of the payload. This relation makes explicit the available internal motion of the system, which can be used to automatically achieve additional tasks. The proposed method does not require to specify a priory the forces in the cables and uses a tension distribution algorithm to optimally distribute them among the robots. The presented framework is also suitable for online teleoperation. Physical simulations with a human-in-the-loop validate the proposed approach.

This paper introduces the CableRobot simulator, which was developed at the Max Planck Institute for Biological Cybernetics in cooperation with the Fraunhofer Institute for Manufacturing Engineering and Automation IPA. The simulator is a completely novel approach to the design of motion simulation platforms in so far as it uses cables and winches for actuation instead of rigid links known from hexapod simulators. This approach allows to reduce the actuated mass, scale up the workspace significantly, and provides great flexibility to switch between system configurations in which the robot can be operated. The simulator will be used for studies in the field of human perception research and virtual reality applications. The paper discusses some of the issues arising from the usage of cables and provides a system overview regarding kinematics and system dynamics as well as giving a brief introduction into possible application use cases.

Haptic guidance has previously been employed to improve human performance in control tasks. This paper presents an experiment to evaluate whether haptic feedback can be used to help humans learn a compensatory tracking task. In the experiment, participants were divided into two groups: the haptic group and the no-aid group. The haptic group performed a first training phase with haptic feedback and a second evaluation phase without haptic feedback. The no-aid group performed the whole experiment without haptic feedback. Results indicated that haptic group achieved better performance than the no-aid group during the training phase. Furthermore, performance of haptic group did not worsen in the evaluation phase when the haptic feedback was turned off. Moreover, the no-aid group needed more experimental trials to achieve similar performance to the haptic group. These findings indicate that haptic feedback helped participants learn the task quicker.

We present a decentralized algorithm for estimating mutual poses (relative positions and orientations) in a group of mobile robots. The algorithm uses relative-bearing measurements, which, for example, can be obtained from onboard cameras, and information about the motion of the robots, such as inertial measurements. It is assumed that all relative-bearing measurements are anonymous; i.e., each specifies a direction along which another robot is located but not its identity. This situation, which is often ignored in the literature, frequently arises in practice and remarkably increases the complexity of the problem. The proposed solution is based on a two-step approach: in the first step, the most likely unscaled relative configurations with identities are computed from anonymous measurements by using geometric arguments, while in the second step, the scale is determined by numeric Bayesian filtering based on the motion model. The solution is first developed for ground robots in SE (2) and then for aerial robots in SE (3). Experiments using Khepera III ground mobile robots and quadrotor aerial robots confirm that the proposed method is effective and robust w.r.t. false positives and negatives of the relative-bearing measuring process.

Mobility of a hexarotor UAV in its standard configuration is limited, since all the propeller force vectors are parallel and they achieve only 4-DoF actuation, similar, e.g., to quadrotors. As a consequence, the hexarotor pose cannot track an arbitrary trajectory while the center of mass is tracking a position trajectory. In this paper, we consider a different hexarotor architecture where propellers are tilted, without the need of any additional hardware. In this way, the hexarotor gains a 6-DoF actuation which allows to independently reach positions and orientations in free space and to be able to exert forces on the environment to resist any wrench for aerial manipulation tasks. After deriving the dynamical model of the proposed hexarotor, we discuss the controllability and the tilt angle optimization to reduce the control effort for the specific task. An exact feedback linearization and decoupling control law is proposed based on the input-output mapping, considering the Jacobian and task acceleration, for non-linear trajectory tracking. The capabilities of our approach are shown by simulation results.

In this paper we introduce the design of a light-weight novel flexible-joint arm for light-weight unmanned aerial vehicles (UAVs), which can be used both for safe physical interaction with the environment and it represents also a preliminary step in the direction of performing quick motions for tasks such as hammering or throwing. The actuator consists of an active pulley driven by a rotational servo motor, a passive pulley which is attached to a rigid link, and the elastic connections (springs) between these two pulleys. We identify the physical parameters of the system, and use an optimal control strategy to maximize its velocity by taking advantage of elastic components. The prototype can be extended to a light-weight variable stiffness actuator. The flexible-joint arm is applied on a quadrotor, to be used in aerial physical interaction tasks, which implies that the elastic components can also be used for stable interaction absorbing the interactive disturbances which might damage the flying system and its hardware. The design is validated through several experiments, and future developments are discussed in the paper.

Identification of landmarks for outdoor navigation is often performed using computationally expensive computer vision methods or via heavy and expensive multi-spectral and range sensors. Both choices are forbidden on Micro Aerial Vehicles (MAV) due to limited payload and computational power. However, an appropriate choice of the hardware sensor equipment allows the employment of mixed multi-spectral analysis and computer vision techniques to identify natural landmarks. In this work, we propose a low-cost low-weight camera array with appropriate optical filters to be exploited both as stereo camera and multi-spectral sensor. Through stereo vision and the Normalized Difference Vegetation Index (NDVI), we are able to classify the observed materials in the scene among several different classes, identify vegetation and water bodies and provide measurements of their relative bearing and distance from the robot. A handheld prototype of this camera array is tested in outdoor environment.

Effectiveness of haptic guidance systems depends on how humans adapt their neuromuscular response to the force feedback. A quantitative insight into adaptation of neuromuscular response can be obtained by identifying neuromuscular dynamics. Since humans are likely to vary their neuromuscular response during realistic control scenarios, there is a need for methods that can identify time-varying neuromuscular dynamics. In this work an identification method is developed which estimates the impulse response of time-varying neuromuscular system by using a Recursive Least Squares (RLS) method. The proposed method extends the commonly used RLS-based method by employing the pseudo inverse operator instead of the inverse operator. This results in improved robustness to external noise. The method was validated in a human in-the-loop experiment. The neuromuscular estimates given by the proposed method were more accurate than those obtained with the commonly used RLS-based method.

The International Air Transport Association forecasts that there will be at least a 30% increase in passenger demand for flights over the next five years. In these circumstances the aircraft industry is looking for new ways to keep passengers occupied, entertained and healthy, and one of the methods under consideration is immersive virtual reality. It is therefore becoming important to understand how motion sickness and presence in virtual reality are influenced by physical motion. We were specifically interested in the use of head-mounted displays (HMD) while experiencing in-flight motions such as turbulence. 50 people were tested in different virtual environments varying in their context (virtual airplane versus magic carpet ride over tropical islands) and the way the physical motion was incorporated into the virtual world (matching visual and auditory stimuli versus no incorporation). Participants were subjected to three brief periods of turbulent motions realized with a motion simulator. Physiological signals (postural stability, heart rate and skin conductance) as well as subjective experiences (sickness and presence questionnaires) were measured. None of our participants experienced severe motion sickness during the experiment and although there were only small differences between conditions we found indications that it is beneficial for both wellbeing and presence to choose a virtual environment in which turbulent motions could be plausible and perceived as part of the scenario. Therefore we can conclude that brief exposure to turbulent motions does not get participants sick.

Virtual reality technology can be considered a multipurpose tool for diverse applications in various domains, for example, training, prototyping, design, entertainment, and research investigating human perception. However, for many of these applications, it is necessary that the designed and computer-generated virtual environments are perceived as a replica of the real world. Many research studies have shown that this is not necessarily the case. Specifically, egocentric distances are underestimated compared to real-world estimates regardless of whether the virtual environment is displayed in a head-mounted display or on an immersive large-screen display. While the main reason for this observed distance underestimation is still unknown, we investigate a potential approach to reduce or even eliminate this distance underestimation. Building up on the angle of declination below the horizon relationship for perceiving egocentric distances, we describe how eye height manipulations in virtual reality should affect perceived distances. In addition, we describe how this relationship could be exploited to reduce distance underestimation for individual users. In a first experiment, we investigate the influence of a manipulated eye height on an action-based measure of egocentric distance perception. We found that eye height manipulations have similar predictable effects on an action-based measure of egocentric distance as we previously observed for a cognitive measure. This might make this approach more useful than other proposed solutions across different scenarios in various domains, for example, for collaborative tasks. In three additional experiments, we investigate the influence of an individualized manipulation of eye height to reduce distance underestimation in a sparse-cue and a rich-cue environment. In these experiments, we demonstrate that a simple eye height manipulation can be used to selectively alter perceived distances on an individual basis, which could be helpful to enable every user to have an experience close to what was intended by the content designer.

In this paper, identification methods are proposed to estimate the neuromuscular and visual responses of a multiloop pilot model. A conventional and widely used technique for simultaneous identification of the neuromuscular and visual systems makes use of cross-spectral density estimates. This paper shows that this technique requires a specific noninterference hypothesis, often implicitly assumed, that may be difficult to meet during actual experimental designs. A mathematical justification of the necessity of the noninterference hypothesis is given. Furthermore, two methods are proposed that do not have the same limitations. The first method is based on autoregressive models with exogenous inputs, whereas the second one combines cross-spectral estimators with interpolation in the frequency domain. The two identification methods are validated by offline simulations and contrasted to the classic method. The results reveal that the classic method fails when the noninterference hypothesis is not fulfilled; on the contrary, the two proposed techniques give reliable estimates. Finally, the three identification methods are applied to experimental data from a closed-loop control task with pilots. The two proposed techniques give comparable estimates, different from those obtained by the classic method. The differences match those found with the simulations. Thus, the two identification methods provide a good alternative to the classic method and make it possible to simultaneously estimate human's neuromuscular and visual responses in cases where the classic method fails.

In this paper the problem of a quadrotor that physically interacts with the surrounding environment through a rigid tool is considered. We present a theoretical design that allows to exert an arbitrary 3D force by using a standard near-hovering controller that was originally developed for contact-free flight control. This is achieved by analytically solving the nonlinear system that relates the quadrotor state, the force exerted by the rigid tool on the environment, and the near-hovering controller action at the equilibrium points, during any generic contact. Stability of the equilibria for the most relevant actions (pushing, releasing, lifting, dropping, and left-right shifting) are proven by means of numerical analysis using the indirect Lyapunov method. An experimental platform, including a suitable tool design, has been developed and used to validate the theory with preliminary experiments.

A new framework for semi-autonomous path planning for mobile robots that extends the classical paradigm of bilateral shared control is presented. The path is represented as a B-spline and the human operator can modify its shape by controlling the motion of a finite number of control points. An autonomous algorithm corrects in real time the human directives in order to facilitate path tracking for the mobile robot and ensures i) collision avoidance, ii) path regularity, and iii) attraction to nearby points of interest. A haptic feedback algorithm processes both human's and autonomous control terms, and their integrals, to provide an information of the mismatch between the path specified by the operator and the one corrected by the autonomous algorithm. The framework is validated with extensive experiments using a quadrotor UAV and a human in the loop with two haptic interfaces.

We present the development of a semi-autonomous quadrotor UAV platform for indoor teleoperation using RGB-D technology as exceroceptive sensor. The platform integrates IMU and Dense Visual Odometry pose estimation in order to stabilize the UAV velocity and track the desired velocity commanded by a remote operator though an haptic interface. While being commanded, the quadrotor autonomously performs a persistent pan-scanning of the surrounding area in order to extend the intrinsically limited field of view. The RGB-D sensor is used also for collision-safe navigation using a probabilistically updated local obstacle map. In the operator visual feedback, pan-scanning movement is real time compensated by an IMU-based adaptive filtering algorithm that lets the operator perform the drive experience in a oscillation-free frame. An additional sensory channel for the operator is provided by the haptic feedback, which is based on the obstacle map and velocity tracking error in order to convey information about the environment and quadrotor state. The effectiveness of the platform is validated by means of experiments performed without the aid of any external positioning system.

In this paper we propose a controller, based on an extension of Interconnection and Damping Assignment-Passivity Based Control (IDA-PBC) framework, for shaping the whole physical characteristics of a quadrotor and for obtaining a desired interactive behavior between the robot and the environment. In the control design, we shape the total energy (kinetic and potential) of the undamped original system by first excluding external effects. In this way we can assign a new dynamics to the system. Then we apply damping injection to the new system for achieving a desired damped behavior. Then we show how to connect a high-level control input to such system by taking advantage of the new desired physics. We support the theory with extensive simulations by changing the overall behavior of the UAV for different desired dynamics, and show the advantage of this method for sliding on a surface tasks, such as ceiling painting, cleaning or surface inspection.

A human-centered design of haptic aids aims at tuning the force feedback based on the effect it has on human behavior. For this goal, a better understanding of the influence of haptic aids on the pilot neuromuscular response becomes crucial. In realistic scenarios, the neuromuscular response can continuously vary depending on many factors, such as environmental factors or pilot fatigue. This paper presents a method that online estimates time-varying neuromuscular dynamics during force-related tasks. This method is based on a Recursive Least Squares (RLS) algorithm and assumes that the neuromuscular response can be approximated by a Finite Impulse Response filter. The reliability and the robustness of the method were investigated by performing a set of Monte-Carlo simulations with increasing level or remnant noise. Even with high level of remnant noise, the RLS algorithm provided accurate estimates when the neuromuscular dynamics were constant or changed slowly. With instantaneous changes, the RLS algorithm needed almost 8s to converge to a reliable estimate. These results seem to indicate that RLS algorithm is a valid tool for estimating online time-varying admittance.

Biodynamic feedthrough (BDFT) is the feedthrough of vehicle accelerations through the human body, leading to involuntary control device inputs. BDFT is a relevant problem as it reduces control performance in a large range of vehicles under various circumstances. This paper proposes an approach to mitigate BDFT. What differentiates this method from other mitigation approaches is that it accounts for adaptations in the neuromuscular dynamics of the human body. It is known that BDFT is strongly dependent on these dynamics. The approach was tested, as proof-of-concept, in an experiment in a motion simulator where participants were asked to fly a simulated vehicle through a virtual tunnel. By evaluating the performance with and without motion disturbance active and with and without cancellation active, the performance of the cancellation approach was evaluated. Results showed that the cancellation approach was successful. The detrimental effects of BDFT, such as a decrease in control performance and increase in control effort, were largely removed.

A biodynamic feedthrough (BDFT) model is proposed that describes how vehicle accelerations feed through the human body, causing involuntary limb motions and so involuntary control inputs. BDFT dynamics strongly depend on limb dynamics, which can vary between persons (between-subject variability), but also within one person over time, e.g., due to the control task performed (within-subject variability). The proposed BDFT model is based on physical neuromuscular principles and is derived from an established admittance model-describing limb dynamics-which was extended to include control device dynamics and account for acceleration effects. The resulting BDFT model serves primarily the purpose of increasing the understanding of the relationship between neuromuscular admittance and biodynamic feedthrough. An added advantage of the proposed model is that its parameters can be estimated using a two-stage approach, making the parameter estimation more robust, as the procedure is largely based on the well documented procedure required for the admittance model. To estimate the parameter values of the BDFT model, data are used from an experiment in which both neuromuscular admittance and biodynamic feedthrough are measured. The quality of the BDFT model is evaluated in the frequency and time domain. Results provide strong evidence that the BDFT model and the proposed method of parameter estimation put forward in this paper allows for accurate BDFT modeling across different subjects (accounting for between-subject variability) and across control tasks (accounting for within-subject variability).

Biodynamic feedthrough (BDFT) occurs when vehicle accelerations feed through the human body and cause involuntary control inputs. This paper proposes a model to quantitatively predict this effect in rotorcraft. This mathematical BDFT model aims to fill the gap between the currently existing black box BDFT models and physical BDFT models. The model structure was systematically constructed using asymptote modeling, a procedure described in detail in this paper. The resulting model can easily be implemented in many typical rotorcraft BDFT studies, using the provided model parameters. The model's performance was validated in both the frequency and time domain. Furthermore, it was compared with several recent BDFT models. The results show that the proposed mathematical model performs better than typical black box models and is easier to parameterize and implement than a recent physical model.

Biodynamic feedthrough (BDFT) is a complex phenomenon, which has been studied for several decades. However, there is little consensus on how to approach the BDFT problem in terms of definitions, nomenclature, and mathematical descriptions. In this paper, a framework for biodynamic feedthrough analysis is presented. The goal of this framework is two-fold. First, it provides some common ground between the seemingly large range of different approaches existing in the BDFT literature. Second, the framework itself allows for gaining new insights into BDFT phenomena. It will be shown how relevant signals can be obtained from measurement, how different BDFT dynamics can be derived from them, and how these different dynamics are related. Using the framework, BDFT can be dissected into several dynamical relationships, each relevant in understanding BDFT phenomena in more detail. The presentation of the BDFT framework is divided into two parts. This paper, Part I, addresses the theoretical foundations of the framework. Part II, which is also published in this issue, addresses the validation of the framework. The work is presented in two separate papers to allow for a detailed discussion of both the framework's theoretical background and its validation.

Biodynamic feedthrough (BDFT) is a complex phenomenon, that has been studied for several decades. However, there is little consensus on how to approach the BDFT problem in terms of definitions, nomenclature, and mathematical descriptions. In this paper, the framework for BDFT analysis, as presented in Part I of this dual publication, is validated and applied. The goal of this framework is twofold. First of all, it provides some common ground between the seemingly large range of different approaches existing in BDFT literature. Secondly, the framework itself allows for gaining new insights into BDFT phenomena. Using recently obtained measurement data, parts of the framework that were not already addressed elsewhere, are validated. As an example of a practical application of the framework, it will be demonstrated how the effects of control device dynamics on BDFT can be understood and accurately predicted. Other ways of employing the framework are illustrated by interpreting the results of three selected studies from the literature using the BDFT framework. The presentation of the BDFT framework is divided into two parts. This paper, Part II, addresses the validation and application of the framework. Part I, which is also published in this journal issue, addresses the theoretical foundations of the framework. The work is presented in two separate papers to allow for a detailed discussion of both the framework's theoretical background and its validation.

For any robot, the ability to recognize and manipulate unknown objects is crucial to successfully work in natural environments. Object recognition and categorization is a very challenging problem, as 3-D objects often give rise to ambiguous, 2-D views. Here, we present a perception-driven exploration and recognition scheme for in-hand object recognition implemented on the iCub humanoid robot. In this setup, the robot actively seeks out object views to optimize the exploration sequence. This is achieved by regarding the object recognition problem as a localization problem. We search for the most likely viewpoint position on the viewsphere of all objects. This problem can be solved efficiently using a particle filter that fuses visual cues with associated motor actions. Based on the state of the filter, we can predict the next best viewpoint after each recognition step by searching for the action that leads to the highest expected information gain. We conduct extensive evaluations of the proposed system in simulation as well as on the actual robot and show the benefit of perception-driven exploration over passive, vision-only processes at discriminating between highly similar objects. We demonstrate that objects are recognized faster and at the same time with a higher accuracy.

We wanted to investigate whether judgments of personality traits based on brief visual observations, namely the 'thin slicing' approach, can also be replicated with biological motion cues consisting of point-light stimuli when different sorts of natural human actions were carried out. We captured six different actions carried out by ten different people using a motion capture system. These actions were then judged in terms of various personal traits by 18 different raters. All actors who recorded those actions were also included as raters. The results showed significant consistency in both the ratings across different raters (inter-rater-reliability) and the ratings between the items (inter-item-reliability). Moreover, accuracy of judgments was also found with self-ratings showing a significant correlation with other people's ratings for a number of different trait judgments.

The ability of humans to apprehend the overall size or volume of an indoor space is not well understood. Previous research has highlighted a 'rectangularity illusion', in which rectangular rooms appear to be larger than square rooms of the same size (identical volume), showing that the subjective perceived space cannot be explained from the mathematical formula for volume, i.e. length × width × height. Instead, the results suggest that one might use the longest dimension of the space as a simplified strategy to assess room size [Sadalla and Oxley 1984].

People use body motion to express and recognise emotions. We investigated whether emotional body expressions can be recognised when they are recorded during natural narration, where actors freely express the emotional colouring of a story told. We then took only the upper body motion trajectories and presented them to participants in the form of animated stick figures. The observers were asked to categorise the emotions expressed in short motion sequences. The results show that recognition level of eleven emotions shown via upper body is significantly above chance level and the responses to motion sequences are consistent across observers.

For teleoperation of multiple mobile robots (e.g., unmanned aerial vehicles), a better situational awareness of remote environments is crucial for good performance. Visual and force feedback are the most common ways to perceive the environments accurately using a vision system and a haptic device, respectively. In outdoor environments, however, it is difficult to use adequate/accurate sensors (e.g., global localization system) for haptic feedback of the remote environments as well as the slave robots. In this paper, we propose a visual and force feedback method to enhance a human operator's situational awareness in multi-robot teleoperation by fabricating a global view of the multi-robot system and transmitting velocity information of one, respectively, using only local information of the robots. The proposed visual feedback method, then, is evaluated via two psychophysical experiments: maneuvering and searching tests. Several measures are also proposed to assess the human operator's maneuverability and situational awareness quantitatively.

In this work we present a novel concept of a quadrotor UAV with tilting propellers. Standard quadrotors are limited in their mobility because of their intrinsic underactuation (only 4 independent control inputs vs. their 6-dof pose in space). The quadrotor prototype discussed in this paper, on the other hand, has the ability to also control the orientation of its 4 propellers, thus making it possible to overcome the aforementioned underactuation and behave as a fully-actuated flying vehicle. We first illustrate the hardware/software specifications of our recently developed prototype, and then report the experimental results of some preliminary, but promising, flight tests which show the capabilities of this new UAV concept.

This paper presents a novel bilateral controller that allows to stably teleoperate the degree of connectivity in the mutual interaction between a remote group of mobile robots considered as the slave-side. A distributed leader-follower scheme allows the human operator to command the overall group motion. The group autonomously maintains the connectivity of the interaction graph by using a decentralized gradient descent approach applied to the Fiedler eigenvalue of a properly weighted Laplacian matrix. The degree of connectivity, and then the flexibility, of the interaction graph can be finely tuned by the human operator through an additional bilateral teleoperation channel. Passivity of the overall system is theoretically proven and extensive human/hardware in-the-loop simulations are presented to empirically validate the theoretical analysis.

For the purpose of autonomous UAV flight control, cameras are ubiquitously exploited as a cheap and effective onboard sensor for obtaining non-metric position or velocity measurements. Since the metric scale cannot be directly recovered from visual input only, several methods have been proposed in the recent literature to overcome this limitation by exploiting independent `metric' information from additional onboard sensors. The flexibility of most approaches is, however, often limited by the need of constantly tracking over time a certain set of features in the environment, thus potentially suffering from possible occlusions or loss of tracking during flight. In this respect, in this paper we address the problem of estimating the scale of the observed linear velocity in the UAV body frame from direct measurement of the instantaneous (and non-metric) optical flow, and the integration of an onboard Inertial Measurement Unit (IMU) for providing (metric) acceleration readings. To this end, two different estimation techniques are developed and critically compared: a standard Extended Kalman Filter (EKF) and a novel nonlinear observer stemming from the adaptive control literature. Results based on simulated and real data recorded during a quadrotor UAV flight demonstrate the effectiveness of the approach.

This article provides longitudinal data for when participants learned to travel with a walking metaphor through virtual reality (VR) worlds, using interfaces that ranged from joystick-only, to linear and omnidirectional treadmills, and actual walking in VR. Three metrics were used: travel time, collisions (a measure of accuracy), and the speed profile. The time that participants required to reach asymptotic performance for traveling, and what that asymptote was, varied considerably between interfaces. In particular, when a world had tight turns (0.75 m corridors), participants who walked were more proficient than those who used a joystick to locomote and turned either physically or with a joystick, even after 10 minutes of training. The speed profile showed that this was caused by participants spending a notable percentage of the time stationary, irrespective of whether or not they frequently played computer games. The study shows how speed profiles can be used to help evaluate participants' proficiency with travel interfaces, highlights the need for training to be structured to addresses specific weaknesses in proficiency (e.g., start-stop movement), and for studies to measure and report that proficiency.

In this paper, we investigate the effect of haptic cueing on a human operator's performance in the field of bilateral teleoperation of multiple mobile robots, particularly multiple unmanned aerial vehicles (UAVs). Two aspects of human performance are deemed important in this area, namely, the maneuverability of mobile robots and the perceptual sensitivity of the remote environment. We introduce metrics that allow us to address these aspects in two psychophysical studies, which are reported here. Three fundamental haptic cue types were evaluated. The Force cue conveys information on the proximity of the commanded trajectory to obstacles in the remote environment. The Velocity cue represents the mismatch between the commanded and actual velocities of the UAVs and can implicitly provide a rich amount of information regarding the actual behavior of the UAVs. Finally, the Velocity+Force cue is a linear combination of the two. Our experimental results show that, while maneuverability is best supported by the Force cue feedback, perceptual sensitivity is best served by the Velocity cue feedback. In addition, we show that large gains in the haptic feedbacks do not always guarantee an enhancement in the teleoperator's performance.

When performing a manual control task, vehicle accelerations can cause involuntary limb motions, which can result in unintentional control inputs. This phenomenon is called biodynamic feedthrough (BDFT). In the past decades, many studies into BDFT have been performed, but its fundamentals are still only poorly understood. What has become clear, though, is that BDFT is a highly complex process, and its occurrence is influenced by many different factors. A particularly challenging topic in BDFT research is the role of the human operator, which is not only a very complex but also a highly adaptive system. In literature, two different ways of measuring and analyzing BDFT are reported. One considers the transfer of accelerations to involuntary forces applied to the control device (CD); the other considers the transfer of accelerations to involuntary CD deflections or positions. The goal of this paper is to describe an approach to unify these two methods. It will be shown how the results of the two methods relate and how this knowledge may aid in understanding BDFT better as a whole. The approach presented is based on the notion that BDFT dynamics can be described by the combination of two transfer dynamics: 1) the transfer dynamics from body accelerations to involuntary forces and 2) the transfer dynamics from forces to CD deflections. The approach was validated using experimental results.

Advanced driving simulators aim at rendering the motion of a vehicle with maximum fidelity, which requires increased mechanical travel, size, and cost of the system. Motion cueing algorithms reduce the motion envelope by taking advantage of limitations in human motion perception, and the most commonly employed method is just to scale down the physical motion. However, little is known on the effects of motion scaling on motion perception and on actual driving performance. This paper presents the results of a European collaborative project, which explored different motion scale factors in a slalom driving task. Three state-of-the-art simulator systems were used, which were capable of generating displacements of several meters. The results of four comparable driving experiments, which were obtained with a total of 65 participants, indicate a preference for motion scale factors below 1, within a wide range of acceptable values (0.4-0.75). Very reduced or absent motion cues significantly degrade driving performance. Applications of this research are discussed for the design of motion systems and cueing algorithms for driving simulation.

Interaction with its environment is a key requisite for a humanoid robot. Especially the ability to recognize and manipulate unknown objects is crucial to successfully work in natural environments. Visual object recognition, however, still remains a challenging problem, as three-dimensional objects often give rise to ambiguous, two-dimensional views. Here, we propose a perception-driven, multisensory exploration and recognition scheme to actively resolve ambiguities that emerge at certain viewpoints. We define an efficient method to acquire two-dimensional views in an object-centered task space and sample characteristic views on a view sphere. Information is accumulated during the recognition process and used to select actions expected to be most beneficial in discriminating similar objects. Besides visual information we take into account proprioceptive information to create more reliable hypotheses. Simulation and real-world results clearly demonstrate the efficiency of active, multisensory exploration over passive, visiononly recognition methods.

We present a decentralized algorithm for estimating mutual 3-D poses in a group of mobile robots, such as a team of UAVs. Our algorithm uses bearing measurements reconstructed, e.g., by a visual sensor, and inertial measurements coming from the robot IMU. Since identification of a specific robot in a group would require visual tagging and may be cumbersome in practice, we simply assume that the bearing measurements are anonymous. The proposed localization method is a non-trivial extension of our previous algorithm for the 2-D case [1], and exhibits similar performance and robustness. An experimental validation of the algorithm has been performed using quadrotor UAVs.

Robot vision became a field of increasing importance in micro aerial vehicle robotics with the availability of small and light hardware. While most approaches rely on external ground stations because of the need of high computational power, we will present a full autonomous setup using only on-board hardware. Our work is based on the continuous homography constraint to recover ego-motion from optical flow. Thus we are able to provide an efficient fall back routine for any kind of UAV (Unmanned Aerial Vehicles) since we rely solely on a monocular camera and on on-board computation. In particular, we devised two variants of the classical continuous 4-point algorithm and provided an extensive experimental evaluation against a known ground truth. The results show that our approach is able to recover the ego-motion of a flying UAV in realistic conditions and by only relying on the limited on-board computational power. Furthermore, we exploited the velocity estimation for closing the loop and controlling the motion of the UAV online.

Standard quadrotor UAVs possess a limited mobility because of their inherent underactuation, i.e., availability of 4 independent control inputs (the 4 propeller spinning velocities) vs. the 6 dofs parameterizing the quadrotor position/ orientation in space. As a consequence, the quadrotor pose cannot track an arbitrary trajectory over time (e.g., it can hover on the spot only when horizontal). In this paper, we propose a novel actuation concept in which the quadrotor propellers are allowed to tilt about their axes w.r.t. the main quadrotor body. This introduces an additional set of 4 control inputs which provides full actuation to the quadrotor position/orientation. After deriving the dynamical model of the proposed quadrotor, we formally discuss its controllability properties and propose a nonlinear trajectory tracking controller based on dynamic feedback linearization techniques. The soundness of our approach is validated by means of simulation results.

In this paper, we present a passivity-based decentralized approach for bilaterally teleoperating a group of UAVs composing the slave side of the teleoperation system. In particular, we explicitly consider the presence of time delays, both among the master and slave, and within UAVs composing the group. Our focus is on analyzing suitable (passive) strategies that allow a stable teloperation of the group despite presence of delays, while still ensuring high flexibility to the group topology (e.g., possibility to autonomously split or join during the motion). The performance and soundness of the approach is validated by means of human/hardware-in-the-loop simulations (HHIL).

Robotic vision has become an important field of research for micro aerial vehicles in the recent years. While many approaches for autonomous visual control of such vehicles rely on powerful ground stations, the increasing availability of small and light hardware allows for the design of more independent systems. In this context, we present a robust algorithm able to recover the UAV ego-motion using a monocular camera and on-board hardware. Our method exploits the continuous homography constraint so as to discriminate among the observed feature points in order to classify those belonging to the dominant plane in the scene. Extensive experiments on a real quadrotor UAV demonstrate that the estimation of the scaled linear velocity in a cluttered environment improved by a factor of 25% compared to previous approaches.

This work extends the framework of bilateral shared control of mobile robots with the aim of increasing the robot autonomy and decreasing the operator commitment. We consider persistent autonomous behaviors where a cyclic motion must be executed by the robot. The human operator is in charge of modifying online some geometric properties of the desired path. This is then autonomously processed by the robot in order to produce an actual path guaranteeing: i) tracking feasibility, ii) collision avoidance with obstacles, iii) closeness to the desired path set by the human operator, and iv) proximity to some points of interest. A force feedback is implemented to inform the human operator of the global deformation of the path rather than using the classical mismatch between desired and executed motion commands. Physically-based simulations, with human/hardware-in-the-loop and a quadrotor UAV as robotic platform, demonstrate the feasibility of the method.

For a quadrotor aircraft, we study the problem of planning a trajectory that connects two arbitrary states while allowing the UAV to grasp a moving target at some intermediate time. To this end, two classes of canonical grasping maneuvers are defined and characterized. A planning strategy relying on differential flatness is then proposed to concatenate one or more grasping maneuvers by means of spline-based subtrajectories, with the additional objective of minimizing the total transfer time. The proposed planning algorithm is not restricted to pure hovering-to-hovering motions and takes into account practical constraints, such as the finite duration of the grasping phase. The effectiveness of the proposed approach is shown by means of physically-based simulations.

The human in manual control of a dynamical system can use both feedback and feedforward control strategies and will select a strategy based on performance and required effort. Literature has shown that feedforward control is used during tracking tasks in response to predictable targets. The influence of an external disturbance signal on the utilization of a feedforward control strategy has never been investigated, however. We hypothesized that the human will use a combined feedforward and feedback control strategy whenever the predictable target signal is sufficiently strong, and a predominantly feedback strategy whenever the random disturbance signal is dominant. From the data of a human-in-the-loop experiment we conclude that feedforward control is used in all the considered experimental conditions, including those where the disturbance signal is dominant and feedforward control does not deliver a marked performance advantage.

A 3D morphable face model represents the complex shape and appearance structure of faces as a compact representation in a vector space. Such a model has proven extremely useful in various fields such as face recognition, facial expression synthesis, face animation, as well as in the perceptual and cognitive sciences. One of the most important steps in creating a 3D morphable model is to establish correspondence between all faces - a highly non-trivial process because of the complex topology of faces. In this paper, we propose to use implicit surfaces for this task, which enables a robust method for establishing dense 3D correspondences. This algorithm is used to build a morphable face model from a large database of Korean faces that were collected. We show that our model is in correspondence with the well-known MPI model so that our face space can be expanded from Asian to Caucasian faces. In addition, with the resulting control parameters of the model, we demonstrate realistic changes of high-level facial attributes.

Biodynamic feedthrough (BDFT) refers to a phenomenon where vehicle accelerations feed through the human body, causing involuntary limb motions, which may cause involuntary control inputs. Many studies have been devoted to mitigating BDFT effects. In the current paper, the effectiveness of a simple, cheap and widely-used hardware component is studied: the armrest. An experiment was conducted in which the BDFT dynamics were measured with and without armrest for different levels of neuromuscular admittance (i.e., different settings of the limb dynamics). The results show that the effect of the armrest on BDFT dynamics varies, both with frequency and neuromuscular admittance.

The realistic depiction of lifelike virtual humans has been the goal of many movie makers in the last decade. Recently, films such as Tron: Legacy and The Curious Case of Benjamin Button have produced highly realistic characters. In the real-time domain, there is also a need to deliver realistic virtual characters, with the increase in popularity of interactive drama video games (such as L.A. Noire™ or Heavy Rain™ ). There have been mixed reactions from audiences to lifelike characters used in movies and games, with some saying that the increased realism highlights subtle imperfections, which can be disturbing. Some developers opt for a stylized rendering (such as cartoon-shading) to avoid a negative reaction [Thompson 2004]. In this paper, we investigate some of the consequences of choosing realistic or stylized rendering in order to provide guidelines for developers for creating appealing virtual characters. We conducted a series of psychophysical experiments to determine whether render style affects how virtual humans are perceived. Motion capture with synchronized eye-tracked data was used throughout to animate custom-made virtual model replicas of the captured actors.

In this paper, a novel decentralized control strategy for bilaterally teleoperating heterogeneous groups of mobile robots from different domains (aerial, ground, marine, and underwater) is proposed. By using a decentralized control architecture, the group of robots, which is treated as the slave side, is made able to navigate in a cluttered environment while avoiding obstacles, interrobot collisions, and following the human motion commands. Simultaneously, the human operator acting on the master side is provided with a suitable force feedback informative of the group response and of the interaction with the surrounding environment. Using passivity-based techniques, we allow the behavior of the group to be as flexible as possible with arbitrary split and join events (e.g., due to interrobot visibility/packet losses or specific task requirements) while guaranteeing the stability of the system. We provide a rigorous analysis of the system stability and steady-state characteristics and validate performance through human/hardware-in-the-loop simulations by considering a heterogeneous fleet of unmanned aerial vehicles (UAVs) and unmanned ground vehicles as a case study. Finally, we also provide an experimental validation with four quadrotor UAVs.

Redirected walking techniques allow people to walk in a larger virtual space than the physical extents of the laboratory. We describe two experiments conducted to investigate human sensitivity to walking on a curved path and to validate a new redirected walking technique. In a psychophysical experiment, we found that sensitivity to walking on a curved path was significantly lower for slower walking speeds (radius of 10 m versus 22 m). In an applied study, we investigated the influence of a velocity-dependent dynamic gain controller and an avatar controller on the average distance that participants were able to freely walk before needing to be reoriented. The mean walked distance was significantly greater in the dynamic gain controller condition, as compared to the static controller (22 m versus 15 m). Our results demonstrate that perceptually motivated dynamic redirected walking techniques, in combination with reorientation techniques, allow for unaided exploration of a large virtual city model.

Rich face models already have a large impact on the fields of computer vision, perception research, as well as computer graphics and animation. Attributes such as descriptiveness, semantics, and intuitive control are desirable properties but hard to achieve. Towards the goal of building such high-quality face models, we present a 3D model-based analysis-by-synthesis approach that is able to parameterize 3D facial surfaces, and that can estimate the state of semantically meaningful components, even from noisy depth data such as that produced by Time-of-Flight (ToF) cameras or devices such as Microsoft Kinect. At the core, we present a specialized 3D morphable model (3DMM) for facial expression analysis and synthesis. In contrast to many other models, our model is derived from a large corpus of localized facial deformations that were recorded as 3D scans from multiple identities. This allows us to analyze unstructured dynamic 3D scan data using a modified Iterative Closest Point model fitting process, followed by a constrained Action Unit model regression, resulting in semantically meaningful facial deformation time courses. We demonstrate the generative capabilities of our 3DMMs for facial surface reconstruction on high and low quality surface data from a ToF camera. The analysis of simultaneous recordings of facial motion using passive stereo and noisy Time-of-Flight camera shows good agreement of the recovered facial semantics.

The use of multiple unmanned aerial vehicles (UAVs) is increasingly being incorporated into a wide range of teleoperation applications. To date, relevant research has largely been focused on the development of appropriate control schemes. In this paper, we extend previous research by investigating how control performance could be improved by providing the teleoperator with haptic feedback cues. First, we describe a control scheme that allows a teleoperator to manipulate the flight of multiple UAVs in a remote environment. Next, we present three designs of haptic cue feedback that could increase the teleoperator's environmental awareness of such a remote environment. These cues are based on the UAVs' i) velocity information, ii) proximity to obstacles, and iii) a combination of these two sources of information. Finally, we present an experimental evaluation of these haptic cue designs. Our evaluation is based on the teleoperator's perceptual sensitivity to the physical environment inhabited by the multiple UAVs. We conclude that a teleoperator's perceptual sensitivity is best served by haptic feedback cues that are based on the velocity information of multiple UAVs.

In this paper, a novel distributed control strategy for teleoperating a fleet of Unmanned Aerial Vehicles (UAVs) is proposed. Using passivity based techniques, we allow the behavior of the UAVs to be as flexible as possible with arbitrary split and join decisions while guaranteeing stability of the system. Furthermore, the overall teleoperation system is also made passive and, therefore, characterized by a stable behavior both in free motion and when interacting with unknown passive obstacles. The performance of the system is validated through semi-experiments.

We propose a novel haptic teleoperation control framework for multiple unmanned aerial vehicles (UAVs) over the Internet, consisting of the three control layers: 1) UAV control layer, where each UAV is abstracted by, and is controlled to follow the trajectory of, its own kinematic virtual point (VP); 2) VP control layer, which modulates each VP's motion according to the teleoperation commands and local artificial potentials (for inter-VP/VP-obstacle collision avoidance and inter-VP connectivity preservation); and 3) teleoperation layer, through which a remote human user can command all (or some) of the VPs' velocity while haptically perceiving the state of all (or some) of the UAVs over the Internet. Master passivity/slave-stability and some asymptotic performance measures are proved. Semi-experiment results are presented to validate the theory.

This paper describes the mechanical and control design of the new 7-DOF CyberMotion Simulator, a redundant industrial manipulator arm consisting of a standard 6 DOF anthropomorphic manipulator plus an actuated cabin attached to the end-effector. Contrarily to Stewart platforms, an industrial manipulator offers several advantages when used as motion simulator: larger motion envelope, higher dexterity, and possibility to realize any end-effector posture within the workspace. In addition to this, the new actuated cabin acts as an additional joint and provides the needed kinematic redundancy to cope with the robot actuator and joint range constraints, which in general can significantly deteriorate the desired motion cues the robot is reproducing. In particular, we will show that, by suitably exploiting the redundancy better results can be obtained in reproducing sustained acceleration cues, a relevant problem when implementing vehicle simulators.

We present a decentralized system for the bilateral teleoperation of groups of UAVs which only relies on relative bearing measurements, i.e., without the need of distance information or global localization. The properties of a 3D bearing-formation are analyzed, and a minimal set of bearings needed for its definition is provided. We also design a novel decentralized formation control almost globally convergent and able to maintain bounded and non-vanishing inter-distances among the agents despite the absence of direct distance measurements. Furthermore, we develop a multi-master/ multi-slave teleoperation setup in order to control the overall behavior of the group and to convey to the human operator suitable force cues, while ensuring stability in presence of delays and packet losses over the master-slave communication channel. The theoretical framework is validated by means of extensive human/hardware in-the-loop simulations using two force-feedback devices and a group of quadrotors.

In this paper, we present an experimental validation of a novel decentralized passivity-based control strategy for teleoperating a group of Unmanned Aerial Vehicles (UAVs): the slave side, consisting of the UAVs, is endowed with large group autonomy by allowing time-varying topology and interrobot/obstacle collision avoidance. The master side, represented by a human operator, controls the group motion and receives suitable force feedback cues informing her/him about the remote slave motion status. Passivity theory is exploited for guaranteeing stability of the slave side and of the overall teleoperation channel. Results of experiments involving the use of 4 quadcopters are reported and discussed, confirming the soundness of the paper theoretical claims.

In this paper, we investigate the maneuverability performance of human teleoperators on multi-robots. First, we propose that maneuverability performance can be assessed by a frequency response function that jointly considers the input force of the operator and the position errors of the multi-robot system that is being maneuvered. Doing so allows us to evaluate maneuverability performance in terms of the human teleoperator's interaction with the controlled system. This allowed us to effectively determine the suitability of different haptic cue algorithms in improving teleoperation maneuverability. Performance metrics based on the human teleoperator's frequency response function indicate that maneuverability performance is best supported by a haptic feedback algorithm which is based on an obstacle avoidance force.

Vehicle accelerations may feed through the human body, causing involuntary limb motions which may lead to involuntary control inputs. This phenomenon is called biodynamic feedthrough (BDFT). Signal cancellation is a possible way of mitigating biodynamic feedthrough. It makes use of a BDFT model to estimate the involuntary control inputs. The BDFT effects are removed by subtracting the modeled estimate of the involuntary control input from the total control signal, containing both voluntary and involuntary components. The success of signal cancellation hinges on the accuracy of the BDFT model used. In this study the potential of signal cancellation is studied by making use of a method called optimal signal cancellation. Here, an identified BDFT model is used off-line to generate an estimate of the involuntary control inputs based on the accelerations present. Results show that reliable signal cancellation requires BDFT models that are both subject and task dependent. The task dependency is of particular importance: failing to adapt the model to changes in the operator's neuromuscular dynamics dramatically decreases the quality of cancellation and can even lead to an increase in unwanted effects. As a reliable and fast on-line identification method of the neuromuscular dynamics of the human operator currently does not exist, real-time signal cancellation is currently not feasible.

The aim of Redirected Walking (RDW) is to redirect a person along their path of travel in a Virtual Environment (VE) in order to increase the virtual space that can be explored in a given tracked area. Among other techniques, the user is redirected on a curved real-world path while visually walking straight in the VE (curvature gain). In this paper, we describe two experiments we conducted to test and extend RDW techniques. In Experiment 1, we measured the effect of walking speed on the detection threshold for curvature of the walking path. In a head-mounted display (HMD) VE, we found a decreased sensitivity for curvature for the slowest walking speed. When participants walked at 0.75 m/s, their detection threshold was approximately 0.1m-1(radius of approximately 10m). In contrast, for faster walking speeds (>;1.0m/s), we found a significantly lower detection threshold of approximately 0.036m-1(radius of approximately 27m). In Experiment 2, we implemented many well known redirection techniques into one dynamic RDW application. We integrated a large virtual city model and investigated RDW for free exploration. Further, we implemented a dynamic RDW controller which made use of the results from Experiment 1 by dynamically adjusting the applied curvature gain depending on the actual walking velocity of the user. In addition, we investigated the possible role of avatars to slow the users down or make them rotate their heads while exploring. Both the dynamic curvature gain controller and the avatar controller were evaluated in Experiment 2. We measured the average distance that was walked before reaching the boundaries of the tracked area. The mean walked distance was significantly larger in the condition where the dynamic gain controller was applied. This distance increased from approximately 15m for static gains to approximately 22m for dynamic gains. This did not come at the cost of an increase in simulator sickness. Applying the avatar controller did reveal an effect on walking distance or simulator sickness.

Despite many recent developments in virtual reality, an effective locomotion interface which allows for normal walking through large virtual environments was until recently still lacking. Here, we describe the new CyberWalk omnidirectional treadmill system, which makes it possible for users to walk endlessly in any direction, while never leaving the confines of the limited walking surface. The treadmill system improves on previous designs, both in its mechanical features and in the control system employed to keep users close to the center of the treadmill. As a result, users are able to start walking, vary their walking speed and direction, and stop walking as they would on a normal, stationary surface. The treadmill system was validated in two experiments, in which both the walking behavior and the performance in a basic spatial updating task were compared to that during normal overground walking. The results suggest that walking on the CyberWalk treadmill is very close to normal walking, especially after some initial familiarization. Moreover, we did not find a detrimental effect of treadmill walking in the spatial updating task. The CyberWalk system constitutes a significant step forward to bringing the real world into the laboratory or workplace.

This study investigated the effect of body-based information (proprioception, etc.) when participants navigated large-scale virtual marketplaces that were either small (Experiment 1) or large in extent (Experiment 2). Extent refers to the size of an environment, whereas scale refers to whether people have to travel through an environment to see the detail necessary for navigation. Each participant was provided with full body-based information (walking through the virtual marketplaces in a large tracking hall or on an omnidirectional treadmill), just the translational component of body-based information (walking on a linear treadmill, but turning with a joystick), just the rotational component (physically turning but using a joystick to translate) or no body-based information (joysticks to translate and rotate). In large and small environments translational body-based information significantly improved the accuracy of participants' cognitive maps, measured using estimates of direction and relative straight line distance but, on its own, rotational body-based information had no effect. In environments of small extent, full body-based information also improved participants' navigational performance. The experiments show that locomotion devices such as linear treadmills would bring substantial benefits to virtual environment applications where large spaces are navigated, and theories of human navigation need to reconsider the contribution made by body-based information, and distinguish between environmental scale and extent.

Selecting a graphical item by pointing with a computer mouse is a ubiquitous task in many graphical user interfaces. Several techniques have been suggested to facilitate this task, for instance, by reducing the required movement distance. Here we measure the natural coordination of eye and mouse pointer control across several search and selection tasks. We find that users automatically minimize the distance to likely targets in an intelligent, task dependent way. When target location is highly predictable, top-down knowledge can enable users to initiate pointer movements prior to target fixation. These findings question the utility of existing assistive pointing techniques and suggest that alternative approaches might be more effective.

This paper considers the problem of realizing a 6-DOF closed-loop motion simulator by exploiting an anthropomorphic serial manipulator as motion platform. Contrary to standard Stewart platforms, an industrial anthropomorphic manipulator offers a considerably larger motion envelope and higher dexterity that let envisage it as a viable and superior alternative. Our work is divided in two papers. In this Part I, we discuss the main challenges in adopting a serial manipulator as motion platform, and thoroughly analyze one key issue: the design of a suitable inverse kinematics scheme for online motion reproduction. Experimental results are proposed to analyze the effectiveness of our approach. Part II [1] will address the design of a motion cueing algorithm tailored to the robot kinematics, and will provide an experimental evaluation on the chosen scenario: closed-loop simulation of a Formula 1 racing car.

This paper, divided in two Parts, considers the problem of realizing a 6-DOF closed-loop motion simulator by exploiting an anthropomorphic serial manipulator as motion platform. After having proposed a suitable inverse kinematics scheme in Part I [1], we address here the other key issue, i.e., devising a motion cueing algorithm tailored to the specific robot motion envelope. An extension of the well-known classical washout filter designed in cylindrical coordinates will provide an effective solution to this problem. The paper will then present a thorough experimental evaluation of the overall architecture (inverse kinematics + motion cueing) on the chosen scenario: closed-loop simulation of a Formula 1 racing car. This will prove the feasibility of our approach in fully exploiting the robot motion capabilities as a motion simulator.

The paper presents an experimental evaluation of two different Haptic aiding concepts: Direct and Indirect Haptic Aiding. Two Haptic systems were designed and tested using an experimental setup. The problem of wind gust rejection in Remotely Piloted Vehicles is used as test bench. Test results show the effectiveness of both methods but a better performance of the IHA-based system for pilots without any previous training about the experiment. DHA-based system provided instead better results after some pilot training on the experiment. Pilots reported better sensation of the wind gusts with IHA-based feedback.

Affordable 3D vision is just about to enter the mass market for consumer products such as video game consoles or TV sets. Having depth information in this context is beneficial for segmentation as well as gaining robustness against illumination effects, both of which are hard problems when dealing with color camera data in typical living room situations. Several techniques compute 3D (or rather 2.5D) depth information from camera data such as realtime stereo, time-of-flight (TOF), or real-time structured light, but all produce noisy depth data at fairly low resolutions. Not surprisingly, most applications are currently limited to basic gesture recognition using the full body. In particular, TOF cameras are a relatively new and promising technology for compact, simple and fast 2.5D depth measurements. Due to the measurement principle of measuring the flight time of infrared light as it bounces off the subject, these devices have comparatively low image resolution (176 x 144 ... 320 x 240 pixels) with a high level of noise present in the raw data.

It is still an unsolved problem how to optimally simulate self-motion using motion simulators. We investigated how a forward acceleration can be simulated as believably as possible on a hexapod motion platform equipped with a projection screen. Human participants rated the believability of brief forward accelerations. These were simulated as visual forward accelerations over a ground plane with people as size cues, presented together with brief forward surge translations and backward pitches of the platform, and synchronous random up--down movements of the camera in the visual scene and the platform. The magnitudes of all of the parameters were varied independently across trials. Even though variability between participants was high, the most believable simulation occurred when visual accelerations were combined with backward pitches of the platform, which changed the gravitoinertial vector direction approximately consistent with the visual acceleration. However, a wide range of platform pitches was accepted as believable. With high visual acceleration cues most participants reported trials as realistic even when the platform tilt rate was above vestibular canal thresholds reported in other works. Other manipulated parameters had only a mild influence on the responses. These results can be used to optimize motion-cueing algorithms for simulating linear accelerations in motion simulators.

Previous work in real and virtual settings has shown that the way in which we interact with objects plays a fundamental role in the way we perceive them. This article uses multidimensional scaling (MDS) analysis to further characterize and quantify the effects of using different haptic exploratory procedures (EPs) on perceptual similarity spaces. In Experiment 1, 20 participants rated similarity on a set of nine novel, 3D objects varying in shape and texture after either following their contours, laterally rubbing their centers, gripping them, or sequentially touching their tips. MDS analysis was used to recover perceptual maps of the objects and relative weights of perceptual dimensions from similarity data. Both the maps and relative weights of shape/texture properties were found to vary as a function of the EP used. In addition, large individual differences in the relative weight of shape/texture were observed. In Experiment 2, 17 of the previous participants repeated Experiment 1 after an average of 105 days. The same patterns of raw similarity ratings, perceptual maps, dimension weights, and individual differences were observed, indicating that perceptual similarities remained stable over time. The findings underscore the role of hand movements and individual biases in shaping haptic perceptual similarity. A framework for validating multimodal virtual displays based on the approach used in the study is also presented.

The CyberWalk is a large size 2D omni-directional platform that allows unconstrained locomotion possibilities to a walking user for VR exploration. In this paper we present the motion control design for the platform, which has been developed within the homonymous European research project. The objective is to compensate the intentional motion of the user, so as to keep her/him always close to the platform center while limiting the perceptual effects due to actuation commands. The controller acts at the acceleration level, using suitable observers to estimate the unmeasurable intentional walker's velocity and acceleration. A moving reference position is used to limit the accelerations felt by the user in critical transients, e.g., when the walker suddenly stops motion. Experimental results are reported that show the benefit of designing separate control gains in the two orthogonal directions (lateral and sagittal) of a frame attached to the walker.

Vection is defined as the compelling sensation of illusory self- motion elicited by a moving sensory, usually visual, stimulus. This paper presents collected introspective data on the experience of linear, circular, and curvilinear vection. We evaluate the differences between twelve different trajectories and the influence of the floor projection on the illusion of self-motion. All of the simulated self- motions examined are of a constant velocity, except for a brief simulated initial acceleration. First, we find that linear translations to the left and right are perceived as the least convincing, while linear down is perceived as the most convincing of the linear trajectories. Second, we find that the floor projection significantly improves the introspective measures of linear vection experienced in a photorealistic three-dimensional town. Finally, we find that while linear forward vection is not perceived to be very convincing, curvilinear forward vection is reported to be as convincing as circular vection. Considering our experimental results, our suggestions for simulators and VE applications where vection is desirable is to increase the number of curvilinear trajectories (as opposed to linear ones) and, if possible, add floor projection in order to improve the illusory sense of self-motion.

Experience indicates that the sense of presence in a virtual environment is enhanced when the participants are able to actively move through it. When exploring a virtual world by walking, the size of the model is usually limited by the size of the available tracking space. A promising way to overcome these limitations are motion compression techniques, which decouple the position in the real and virtual world by introducing imperceptible visual-proprioceptive conflicts. Such techniques usually precalculate the redirection factors, greatly reducing their robustness. We propose a novel way to determine the instantaneous rotational gains using a controller based on an optimization problem. We present a psychophysical study that measures the sensitivity of visual-proprioceptive conflicts during walking and use this to calibrate a real-time controller. We show the validity of our approach by allowing users to walk through virtual environments vastly larger than the tracking space.

No abstract available.

The human face is capable of producing an astonishing variety of expressions—expressions for which sometimes the smallest difference changes the perceived meaning considerably. Producing realistic-looking facial animations that are able to transmit this degree of complexity continues to be a challenging research topic in computer graphics. One important question that remains to be answered is: When are facial animations good enough? Here we present an integrated framework in which psychophysical experiments are used in a first step to systematically evaluate theperceptualquality of several different computer-generated animations with respect to real-world video sequences. The first experiment provides an evaluation of several animation techniques, exposing specific animation parameters that are important to achieve perceptual fidelity. In a second experiment, we then use these benchmarked animation techniques in the context of perceptual research in order to systematically investigate the spatiotemporal characteristics of expressions. A third and final experiment uses the quality measures that were developed in the first two experiments to examine the perceptual impact of changing facial features to improve the animation techniques. Using such an integrated approach, we are able to provide important insights into facial expressions for both the perceptual and computer graphics community.

Categorization of scenes is a fundamental process of human vision that allows us to efficiently and rapidly analyze our surroundings. Several studies have explored the processes underlying human scene categorization, but they have focused on processing global image information. In this study, we present both psychophysical and computational experiments that investigate the role of local versus global image information in scene categorization. In a first set of human experiments, categorization performance is tested when only local or only global image information is present. Our results suggest that humans rely on local, region-based information as much as on global, configural information. In addition, humans seem to integrate both types of information for intact scene categorization. In a set of computational experiments, human performance is compared to two state-of-the-art computer vision approaches that have been shown to be psychophysically plausible and that model either local or global information. In addition to the influence of local versus global information, in a second series of experiments, we investigated the effect of color on the categorization performance of both the human observers and the computational model. Analysis of the human data suggests that color is an additional channel of perceptual information that leads to higher categorization results at the expense of increased reaction times in the intact condition. However, it does not affect reaction times when only local information is present. When color is removed, the employed computational model follows the relative performance decrease of human observers for each scene category and can thus be seen as a perceptually plausible model for human scene categorization based on local image information.

The goal of stylization is to provide an abstracted representation of an image that highlights specific types of visual information. Recent advances in computer graphics techniques have made it possible to render many varieties of stylized imagery efficiently making stylization into a useful technique, not only for artistic, but also for visualization applications. In this paper, we report results from two sets of experiments that aim at characterizing the perceptual impact and effectiveness of three different stylization techniques in the context of dynamic facial expressions. In the first set of experiments, animated facial expressions are stylized using three common techniques (brush, cartoon, and illustrative stylization) and investigated using different experimental measures. Going beyond the usual questionnaire approach, these experiments compare the techniques according to several criteria ranging from subjective preference to task-dependent measures (such as recognizability, intensity) allowing us to compare behavioral and introspective approaches. The second set of experiments use the same stylization techniques on real-world video sequences in order to compare the effect of stylization on natural and artificial stimuli. Our results shed light on how stylization of image contents affects the perception and subjective evaluation of both real and computer-generated facial expressions.

The development of high dynamic range (HDR) imagery has brought us to the verge of arguably the largest change in image display technologies since the transition from black-and-white to color television. Novel capture and display hardware will soon enable consumers to enjoy the HDR experience in their own homes. The question remains, however, of what to do with existing images and movies, which are intrinsically low dynamic range (LDR). Can this enormous volume of legacy content also be displayed effectively on HDR displays? We have carried out a series of rigorous psychophysical investigations to determine how LDR images are best displayed on a state-of-the-art HDR monitor, and to identify which stages of the HDR imaging pipeline are perceptually most critical. Our main findings are: (1) As expected, HDR displays outperform LDR ones. (2) Surprisingly, HDR images that are tone-mapped for display on standard monitors are often no better than the best single LDR exposure from a bracketed sequence. (3) Most importantly of all, LDR data does not necessarily require sophisticated treatment to produce a compelling HDR experience. Simply boosting the range of an LDR image linearly to fit the HDR display can equal or even surpass the appearance of a true HDR image. Thus the potentially tricky process of inverse tone mapping can be largely circumvented.

Virtual Reality (VR) is increasingly being used in industry, medicine, entertainment, education, and research. It is generally critical that the VR setups produce behavior that closely resembles real world behavior. One part of any task is the ability to control our posture. Since postural control is well studied in the real world and is known to be strongly influenced by visual information, it is an ideal metric for examining the behavioral fidelity of VR setups. Moreover, VR-based experiments on postural control can provide fundamental new insights into human perception and cognition. Here, we employ the "swinging room paradigm" to validate a specific VR setup. Furthermore, we systematically examined a larger range of room oscillations than previously studied in any single setup. We also introduce several new methods and analyses that were specifically designed to optimize the detection of synchronous swinging between the observer and the virtual room. The results show that the VR setup has a very high behavioral fidelity and that increases in swinging room amplitude continue to produce increases in body sway even at very large room displacements (+/- 80 cm). Finally, the combination of new methods proved to be a very robust, reliable, and sensitive way of measuring body sway.

We attempt to shed light on the algorithms humans use to classify images of human faces according to their gender. For this, a novel methodology combining human psychophysics and machine learning is introduced. We proceed as follows. First, we apply principal component analysis (PCA) on the pixel information of the face stimuli. We then obtain a data set composed of these PCA eigenvectors combined with the subjects' gender estimates of the corresponding stimuli. Second, we model the gender classification process on this data set using a separating hyperplane (SH) between both classes. This SH is computed using algorithms from machine learning: the support vector machine (SVM), the relevance vector machine, the prototype classifier, and the K-means classifier. The classification behavior of humans and machines is then analyzed in three steps. First, the classification errors of humans and machines are compared for the various classifiers, and we also assess how well machines can recreate the subjects' internal decision boundary by studying the training errors of the machines. Second, we study the correlations between the rank-order of the subjects' responses to each stimulus-the gender estimate with its reaction time and confidence rating-and the rank-order of the distance of these stimuli to the SH. Finally, we attempt to compare the metric of the representations used by humans and machines for classification by relating the subjects' gender estimate of each stimulus and the distance of this stimulus to the SH. While we show that the classification error alone is not a sufficient selection criterion between the different algorithms humans might use to classify face stimuli, the distance of these stimuli to the SH is shown to capture essentials of the internal decision space of humans. Furthermore, algorithms such as the prototype classifier using stimuli in the center of the classes are shown to be less adapted to model human classification behavior than algorithms such as the SVM based on stimuli close to the boundary between the classes.

The perceived similarity between objects may well vary according to the sensory modality/modalities in which they are experienced, an important consideration for the design of multimodal interfaces. In this study, we present a similarity-based method for comparing the perceptual importance of object properties in touch and in vision and show how the method can also be used to validate computational measures of object properties. Using either vision or touch, human subjects judged the similarity between novel, 3D objects which varied parametrically in shape and texture. Similarities were also computed using a set of state-of-the art 2D and 3D computational measures. Two resolutions of 2D and 3D object data were used for these computations in order to test for scale dependencies. Multidimensional scaling (MDS) was then performed on all similarity data, yielding maps of the stimuli in both perceptual and computational spaces, as well as the relative weight of shape and texture dimensions. For this object set, we found that visual subjects accorded more importance to shape than texture, while haptic subjects weighted them roughly evenly. Fit errors between human and computational maps were then calculated to assess each feature's perceptual validity. Shape-biased features provided good overall fits to the human visual data; however, no single feature yielded a good overall fit to the haptic data, in which we observed large individual differences. This work demonstrates how MDS techniques can be used to evaluate computational object features using the criterion of perceptual similarity. It also demonstrates a way of assessing how the perceptual validity of a feature varies as a function of parameters such as the target modality and the resolution of object data. Potential applications of this method for the design of unimodal and multimodal human---machine interfaces are discussed.

Research on self-motion perception and simulation has traditionally focused on the contribution of physical stimulus properties (“bottom-up factors”) using abstract stimuli. Here, we demonstrate that cognitive (“top-down”) mechanisms like ecological relevance and presence evoked by a virtual environment can also enhance visually induced self-motion illusions (vection). In two experiments, naive observers were asked to rate presence and the onset, intensity, and convincingness of circular vection induced by different rotating visual stimuli presented on a curved projection screen (FOV: 54° × 45°). Globally consistent stimuli depicting a natural 3D scene proved more effective in inducing vection and presence than inconsistent (scrambled) or unnatural (upside-down) stimuli with similar physical stimulus properties. Correlation analyses suggest a direct relationship between spatial presence and vection. We propose that the coherent pictorial depth cues and the spatial reference frame evoked by the naturalistic environment increased the believability of the visual stimulus, such that it was more easily accepted as a stable “scene” with respect to which visual motion is more likely to be judged as self-motion than object motion. This work extends our understanding of mechanisms underlying self-motion perception and might thus help to improve the effectiveness and believability of virtual reality applications.

Photo editing software allows digital images to be blurred, warped or re-colored at the touch of a button. However, it is not currently possible to change the material appearance of an object except by painstakingly painting over the appropriate pixels. Here we present a method for automatically replacing one material with another, completely different material, starting with only a single high dynamic range image as input. Our approach exploits the fact that human vision is surprisingly tolerant of certain (sometimes enormous) physical inaccuracies, while being sensitive to others. By adjusting our simulations to be careful about those aspects to which the human visual system is sensitive, we are for the first time able to demonstrate significant material changes on the basis of a single photograph as input.

Do we judge similarity between two objects to be the same using touch and vision? We investigated this using psychophysical experiments in which subjects rated the similarity between objects presented either visually or haptically. The stimuli were a family of novel, three-dimensional objects whose microgeometry ("texture") and macrogeometry ("shape") were parametrically varied. Multidimensional scaling of the similarity data was used to reconstruct haptic and visual perceptual spaces. For both modalities, a two-dimensional perceptual space was found whose dimensions clearly corresponded to shape and texture. Interestingly, shape dominated in visual space, whereas both shape and texture were important in haptic space. Furthermore, stimuli clusters were observed in this space, suggesting the emergence of category structure based on similarity relationships. The same category boundaries were confirmed in a visual free sorting experiment. This study reveals differences in object processing across modality and demonstrates an approach for analyzing such differences in multisensory visualizations.

Despite recent technological advances, convincing self-motion simulation in virtual reality (VR) is difficult to achieve, and users often suffer from motion sickness and/or disorientation in the simulated world. Instead of trying to simulate self-motions with physical realism (as is often done for, e.g., driving or flight simulators), we propose in this paper a perceptually oriented approach towards self-motion simulation. Following this paradigm, we performed a series of psychophysical experiments to determine essential visual, auditory, and vestibular/tactile parameters for an effective and perceptually convincing self-motion simulation. These studies are a first step towards our overall goal of achieving lean and elegant self-motion simulation in virtual reality (VR) without physically moving the observer. In a series of psychophysical experiments about the self-motion illusion (circular vection), we found that (i) vection as well as presence in the simulated environment is increased by a consistent, naturalistic visual scene when compared to a sliced, inconsistent version of the identical scene, (ii) barely noticeable marks on the projection screen can increase vection as well as presence in an unobtrusive manner, (iii) physical vibrations of the observer's seat can enhance the vection illusion, and (iv) spatialized 3D audio cues embedded in the simulated environment increase the sensation of self-motion and presence. We conclude that providing consistent cues about self-motion to multiple sensory modalities can enhance vection, even if physical motion cues are absent. These results yield important implications for the design of lean and elegant self-motion simulators.

Communication plays a central role in everday life. During an average conversation, information is exchanged in a variety of ways, including through facial motion. Here, we employ a custom, model-based image manipulation technique to selectively “freez ” portions of a face in video recordings in order to determine the areas that are sufficient for proper recognition of nine conversational expressions. The results show that most expressions rely primarily on a single facial area to convey meaning with different expressions using different areas. The results also show that the combination of rigid head, eye, eyebrow, and mouth motions is sufficient to produce expressions that are as easy to recognize as the original, unmanipulated recordings. Finally, the results show that the manipulation technique introduced few perceptible artifacts into the altered video sequences. This fusion of psychophysics and computer graphics techniques provides not only fundamental insights into human perception and cognition, but also yields the basis for a systematic description of what needs to move in order to produce realistic, recognizable conversational facial animations.

When light strikes a translucent material (such as wax, milk or fruit flesh), it enters the body of the object, scatters and reemerges from the surface. The diffusion of light through translucent materials gives them a characteristic visual softness and glow. What image properties underlie this distinctive appearance? What cues allow us to tell whether a surface is translucent or opaque? Previous work on the perception of semitransparent materials was based on a very restricted physical model of thin filters [Metelli 1970; 1974a,b]. However, recent advances in computer graphics [Jensen et al. 2001; Jensen and Buhler 2002] allow us to efficiently simulate the complex subsurface light transport effects that occur in real translucent objects. Here we use this model to study the perception of translucency, using a combination of psychophysics and image statistics. We find that many of the cues that were traditionally thought to be important for semitransparent filters (e.g., X-junctions) are not relevant for solid translucent objects. We discuss the role of highlights, color, object size, contrast, blur, and lighting direction in the perception of translucency. We argue that the physics of translucency are too complex for the visual system to estimate intrinsic physical parameters by inverse optics. Instead, we suggest that we identify translucent materials by parsing them into key regions and by gathering image statistics from these regions.

“Spatial updating” refers to the process that automatically updates our egocentric mental representation of our immediate surround during self-motions, which is essential for quick and robust spatial orientation. To investigate the relative contribution of visual and vestibular cues to spatial updating, two experiments were performed in a high-end Virtual Reality system. Participants were seated on a motion platform and saw either the surrounding room or a photorealistic virtual model presented via head-mounted display or projection screen. After upright rotations, participants had to point “as accurately and quickly as possibl ” to previously learned targets that were outside of the current field of view (FOV). Spatial updating performance, quantified as response time, configuration error, and pointing error, was comparable in the real and virtual reality conditions when the FOV was matched. Two further results challenge the prevailing basic assumptions about spatial updating: First, automatic, reflexlike spatial updating occurred without any physical motion, i.e., visual information from a known scene alone can , indeed, be sufficient, especially for large FOVs. Second, continuous-motion information is not, in fact, mandatory for spatial updating---merely presenting static images of new orientations proved sufficient, which motivated our distinction between continuous and instant-based spatial updating.

We study gender discrimination of human faces using a combination of psychophysical classification and discrimination experiments together with methods from machine learning. We reduce the dimensionality of a set of face images using principal component analysis, and then train a set of linear classifiers on this reduced representation (linear support vec- tor machines (SVMs), relevance vector machines (RVMs), Fisher linear discriminant (FLD), and prototype (prot) classifiers) using human clas- sification data. Because we combine a linear preprocessor with linear classifiers, the entire system acts as a linear classifier, allowing us to visu- alise the decision-image corresponding to the normal vector of the separ- ating hyperplanes (SH) of each classifier. We predict that the female-to- maleness transition along the normal vector for classifiers closely mim- icking human classification (SVM and RVM [1]) should be faster than the transition along any other direction. A psychophysical discrimina- tion experiment using the decision images as stimuli is consistent with this prediction.

Abstract Facial expressions can be used to direct the flow of a conversation as well as to improve the clarity of communication. The critical physical differences between expressions can, however, be small and subtle. Clear presentation of facial expressions in applied settings, then, would seem to require a large conversational agent. Given that visual displays are generally limited in size, the usage of a large conversational agent would reduce the amount of space available for the display of other information. Here, we examine the role of image size in the recognition of facial expressions. The results show that conversational facial expressions can be easily recognized at surprisingly small image sizes. Copyright © 2004 John Wiley & Sons, Ltd.

No abstract available.

Regardless of whether the humans involved are virtual or real, well-developed conversational skills are a necessity. The synthesis of interface agents that are not only understandable but also believable can be greatly aided by knowledge of which facial motions are perceptually necessary and sufficient for clear and believable conversational facial expressions. Here, we recorded several core conversational expressions (agreement, disagreement, happiness, sadness, thinking, and confusion) from several individuals, and then psychophysically determined the perceptual ambiguity and believability of the expressions. The results show that people can identify these expressions quite well, although there are some systematic patterns of confusion. People were also very confident of their identifications and found the expressions to be rather believable. The specific pattern of confusions and confidence ratings have strong implications for conversational animation. Finally, the present results provide the information necessary to begin a more fine-grained analysis of the core components of these expressions.

It is well known that the human visual system can reconstruct depth from simple random-dot displays given binocular disparity or motion information. This fact has lent support to the notion that stereo and structure from motion systems rely on low-level primitives derived from image intensities. In contrast, the judgment of surface transparency is often considered to be a higher-level visual process that, in addition to pictorial cues, utilizes stereo and motion information to separate the transparent from the opaque parts. We describe a new illusion and present psychophysical results that question this sequential view by showing that depth from transparency and opacity can override the bias to see rigid motion. The brain's computation of transparency may involve a two-way interaction with the computation of structure from motion.

We describe a parallel algorithm for computing optical flow from short-range motion. Regularizing optical flow computation leads to a forruulation which minimizes matching error and, at the same time, maximises smoothness of the optical flow. We develop an approximation to the full regularization computation in which corresponding points are found by comparing local patches of the images. Selection aniong competing matches is performed using a winner-take-all scheme. The algorithm accommodates many different image transformations uniformly, with siniilar results, from brightness to edges. The optical flow computed froni different image transformations, such as edge detection and direct brightness computation, can be simply combined. The algorithm is easily implemented using local operations on a finegrained computer, and has been implemented on a Connection Machine. Experiments with natural images show that the scheme is effective and robust against noise. The algorithm leads to dense optical flow fields ; in addition, inforniation from matching facilitates segmentation.