Assistive devices are to be designed with the objective of use in daily-life as well as broad adoption by end users. In this context, it is necessary to tackle usability challenges by properly detecting and acting in accordance to user intents while minimizing the device installation complexity as well. In the case of physical assistive devices, using force/torque sensors is advantageous to detect user intent compared to EMG interfaces, but it remains difficult to correctly translate the detected intent into actuator motions. Focusing on upper-limb assistive robots, the user voluntary force is commonly used with a controller based on an admittance approach which leads to relatively poor reactivity and requires the user to develop force throughout the movement which can lead to fatigue, particularly for people with upper-limb impairments. This work proposes a Force-Triggered (FT) controller which can initiate and maintain movement only from short force impulses. The user voluntary forces are retrieved from total interaction forces by subtracting the passive component measured beforehand during a calibration phase. This paper presents the design of the proposed FT controller and its preliminary testing on pick-and-place tasks compared to an admittance strategy. This experiment was performed with one participant without impairment, equipped with an upper-limb exoskeleton prototype designed from recommendations of physical medicine therapists. This preliminary work highlights the potential of the proposed FT controller. Also, it provides directions for future work and clinical trials with end-users to assess the proposed FT approach usability while used alone or in the form of an hybrid controller between FT and admittance strategies.

Immersive environments provide opportunities to learn and transfer skills to real life. This opens up new areas of application, such as rehabilitation, where people with neurological disabilities can learn to drive a power wheelchair (PWC) through the development of immersive simulators. To expose these specific users to daily-life study interaction situations, it is important to ensure realistic interactions with the virtual humans that populate the simulated environment, as PWC users should learn to drive and navigate under everyday conditions. While non-verbal pedestrian-pedestrian interactions have been extensively studied, understanding pedestrian-PWC user interactions during locomotion is still an open research area. Our study aimed to investigate the regulation of interpersonal distance (i.e., proxemics) between a pedestrian and a PWC user in real and virtual situations. We designed 2 experiments in which 1) participants had to reach a goal by walking (respectively driving a PWC) and avoid a static PWC confederate (respectively a standing confederate) and 2) participants had to walk to a goal and avoid a static confederate seated on a PWC in real and virtual conditions. Our results showed that interpersonal distances were significantly different whether the pedestrian avoided the PWC user or vice versa. We also showed an influence of the orientation of the person to be avoided. We discuss these findings with respect to pedestrian-pedestrian interactions, as well as their implications for the design of virtual humans interacting with PWC users for rehabilitation applications. In particular, we proposed a proof of concept by adapting existing microscopic crowd simulation algorithms to consider the specificity of pedestrian-PWC user interactions.

Rich, informative and realistic haptic feedback is key to enhancing Virtual Reality (VR) manipulation. Tangible objects provide convincing grasping and manipulation interactions with haptic feedback of e.g., shape, mass and texture properties. But these properties are static, and cannot respond to interactions in the virtual environment. On the other hand, vibrotactile feedback provides the opportunity for delivering dynamic cues rendering many different contact properties, such as impacts, object vibrations or textures. Handheld objects or controllers in VR are usually restricted to vibrating in a monolithic fashion. In this article, we investigate how spatialiazing vibrotactile cues within handheld tangibles could enable a wider range of sensations and interactions. We conduct a set of perception studies, investigating the extent to which spatialization of vibrotactile feedback within tangible objects is possible as well as the benefits of proposed rendering schemes leveraging multiple actuators in VR. Results show that vibrotactile cues from localized actuators can be discriminated and are beneficial for certain rendering schemes.

This paper presents a generic power wheelchair dynamic model. As a first contribution, this paper proposes to use a generic model composed of a geometric model and a lumped model in order to be compliant with a wide range of existing commercially available wheelchairs. In this model, a set of essential parameters are enough to accurately replicate the dynamic behavior of a wheelchair. As a second contribution, this paper presents an identification method of a n-wheel type power wheelchair. The presented model is restricted to the sagittal plane only, which is sufficient to study the reliability of the identification and validation methods. Moreover, a Motion Cueing Algorithm based on the proposed model controls a simulator mechanical platform. The generic model has been then validated through a user study with 18 able-bodied participants evaluating the self-motion perception with our multisensory power wheelchair driving simulator. Results show that the simplified model is sufficient to provide accurate sensations to the user with respect to their experience while driving a power wheelchair.

The evaluation of robot capabilities to navigate human crowds is essential to conceive new robots intended to operate in public spaces. This paper initiates the development of a benchmark tool to evaluate such capabilities; our long term vision is to provide the community with a simulation tool that generates virtual crowded environment to test robots, to establish standard scenarios and metrics to evaluate navigation techniques in terms of safety and efficiency, and thus, to install new methods to benchmarking robots’ crowd navigation capabilities. This paper presents the architecture of the simulation tools, introduces first scenarios and evaluation metrics, as well as early results to demonstrate that our solution is relevant to be used as a benchmark tool.

In the case of non-holonomic robot navigation, path planning algorithms such as Rapidly-exploring Random Tree (RRT) rarely provide feasible and smooth paths without the need of additional processing. Furthermore, in a transport context like power wheelchair navigation, passenger comfort should be a priority and influence path planning strategy. In this paper, we propose a local path planner which guarantees bounded curvature value and continuous Cubic Bézier piecewise curves connection. To simulate and test this Cubic Bézier local path planner, we developed a new RRT version (CBB-RRT*) which generates on-the fly comfortable path adapted to non-holonomic constraints.

Power wheelchairs are one of the main solutions for people with reduced mobility to maintain or regain autonomy and a comfortable and fulfilling life. However, driving a power wheelchair in a safe way is a difficult task that often requires training methods based on real-life situations. Although these methods are widely used in occupational therapy, they are often too complex to implement and unsuitable for some people with major difficulties. In this context, we collaborated with clinicians to develop a Virtual Reality based power wheelchair simulator. This simulator is an innovative training tool adapted to any type of situations and impairments. In this paper, we present a clinical study in which 29 power wheelchair regular users were asked to complete a clinically validated task designed by clinicians within two conditions: driving in a virtual environment with our simulator and driving in real conditions with a real power wheelchair. The objective of this study is to compare performances between the two conditions and to evaluate the Quality of Experience provided by our simulator in terms of Sense of Presence and Cybersickness. Results show that participants complete the tasks in a similar amount of time for both real and virtual conditions, using respectively a real power wheelchair and our simulator. Results also show that our simulator provides a high level of Sense of Presence and provokes only slight to moderate Cybersickness discomforts resulting in a valuable Quality of Experience.

Autonomy and the ability to maintain social activities can be challenging for people with disabilities experiencing reduced mobility. In the case of disabilities that impact mobility, power wheelchairs can help such people retain or regain autonomy. Nonetheless, driving a power wheelchair is a complex task that requires a combination of cognitive, visual and visuo-spatial abilities. In practice, people need to pass prior ability tests and driving training before being prescribed a power wheelchair by their therapist. Still, conventional training in occupational therapy can be insufficient for some people with severe cognitive and/or visuo-spatial functions. As such, these people are often prevented from obtaining a power wheelchair prescription from their therapist due to safety concerns. In this context, driving simulators might be efficient and promising tools to provide alternative, adaptive, flexible, and safe training. In previous work, we proposed a Virtual Reality (VR) driving simulator integrating vestibular feedback to simulate wheelchair motion sensations. The performance and acceptability of a VR simulator rely on satisfying user Quality of Experience (QoE). Therefore, our simulator is designed to give the user a high Sense of Presence (SoP) and low Cybersickness. This paper presents a pilot study assessing the impact of the vestibular feedback provied on user QoE. Participants were asked to perform a driving task whilst in the simulator under two conditions: with and without vestibular feedback. User QoE is assessed through subjective questionnaires measuring user SoP and cybersickness. The results show that vestibular feedback activation increases SoP and decreases cybersickness. This study constitutes a mandatory step before clinical trials and, as such, only enrolled people without disabilities.

This paper explores the use of Virtual Reality (VR) to study humanrobot interactions during navigation tasks by both immersing a user and a robot in a shared virtual spaces. VR combines the advantages of being safe (as robots and humans interacting by the means of VR but can physically be in remote places) and ecological (realistic environments are perceived by the robot and the human, and natural behaviors can be observed). Nevertheless, VR can introduce perceptual biases in the interaction and affect in some ways the observed behaviors, which can be problematic when used to acquire experimental data. In our case, not only human perception is concerned, but also the one of the robot which requires to be simulated to perceive the VR world. Thus, the contribution of this paper is twofold. It first provides a technical solution to perform human robot interactions in navigation tasks through VR: we describe how we combine motion tracking, VR devices, as well as robot sensors simulation algorithms to immerse together a human and a robot in a shared virtual space. We then assess a simple interaction task that we replicate in real and in virtual conditions to perform a first estimation of the importance of the biases introduced by the use of VR on both a Human and a robot. Our conclusions are in favor of using VR to study human-robot interactions, and we are developing directions for future work.

Needle insertion procedures can greatly benefit from robotic systems to improve their accuracy and success rate. However, a fully automated system is usually not desirable and the clinicians need to be included in the control loop. In this paper we present a teleoperation framework for beveledtip flexible needle steering that enables the user to directly and intuitively control the trajectory of the needle tip via a haptic interface. The 6 degrees of freedom of the needle base are used to perform several automatic safety and targeting tasks in addition to the one controlled by the user. Real-time visual feedback is provided by a 3D ultrasound probe and used to track the 3D location of the needle and of a spherical target. Several haptic force feedback are compared as well as two different levels of mix between automated and user-controlled tasks. A validation of the framework is conducted in gelatin phantom and a mean targeting accuracy of 2.5 mm is achieved. The results show that providing an adequate haptic guidance to the user can reduce the risks of damage to the tissues while still letting the surgeon in control of the tip trajectory.

Power wheelchairs allow people with motor disabilities to have more mobility and independence. However, driving safely such a vehicle is a daily challenge particularly in urban environments while navigating on sidewalks, negotiating curbs or dealing with uneven grounds. Indeed, differences of elevation have been reported to be one of the most challenging environmental barrier to negotiate, with tipping and falling being the most common accidents power wheelchair users encounter. It is thus our challenge to design assistive solutions for power wheelchair navigation in order to improve safety while navigating in such environments. To this aim, we propose a shared-control algorithm which provides assistance while navigating with a wheelchair in an environment consisting of negative obstacles. We designed a dedicated sensor-based control law allowing trajectory correction while approaching negative obstacles e.g. steps, curbs, descending slopes. This shared control proposed method takes into account the human-in-the loop factor. In this study, our solution the ability of our system to ensure a safe trajectory while navigating on a sidewalk is demonstrated through simulation, thus providing a proof-of-concept of our method.

Smart powered wheelchairs can increase mobility and independence for people with disability by providing navigation support. This support can be supplied in the form of autonomous or semi-autonomous obstacle avoidance systems. However, for rehabilitation or learning purposes, it would be of great benefit for wheelchair users to have a better understanding of the surrounding environment while driving. Therefore, another way of providing navigation support is to communicate information through a dedicated and adapted feedback interface. We here propose a framework in which feedback is provided by sending forces through the wheelchair controller as the user steers the wheelchair. This solution is based on a low complex optimization framework able to perform smooth trajectory correction and to provide obstacle avoidance. The impact of the proposed haptic guidance solution on user driving performance was assessed during this pilot study for validation purposes through an experiment with 4 able-bodied participants. They were asked to drive a power wheelchair on an obstacle course with and without activation of the force feedback. Results of this pilot study showed that the number of collisions significantly decreased while force feedback was activated, thus validating the proposed framework.

Works on robotic needle steering often consider either tip-based control of flexible beveled tip needles or control of the bending of symmetric tip needles. In this paper a control law for needle steering which uses both direct manipulation of the needle base and the control of the tip deflection, is proposed. A 3D model of a beveled tip needle is used to determine the base motion that is needed to obtain the desired tip motion. The needle-tissue interaction model uses local virtual springs placed along the needle shaft and takes into account the interaction forces at the bevel. Online estimation and update of this model parameters is performed via visual feedback. The low level controller uses the task function framework to allow control of the tip velocity. Additionally duty cycling method is used if a reduction of the natural deflection of the needle tip is needed. Finally an experimental targeting task in a gelatin phantom is presented for preliminary validation of the framework. The method provided sub-millimeter accuracy on a target that would be unreachable using only tip-based control.

With the increasing number of clinical interventions using needle shaped tools, robotic control of needle insertion procedures has been an active research field for many years. In this work we propose a 3D model of a flexible needle that takes into account tissue deformations in order to predict the needle shape and trajectory when it is inserted using a robotic arm. To account for tissue displacements, we designed a method based on visual feedback that updates the interaction model between the needle and the tissue using an unscented Kalman filter. Results obtained from several needle insertions in a soft tissue phantom showed that the method gives good performance in terms of needle trajectory prediction. This model was also considered in a closed-loop control approach to allow automatic reaching of a target.

Motor or visual impairments may prevent a user from steering a wheelchair effectively in indoor environments. In such cases, joystick jerks arising from uncontrolled motions may lead to collisions with obstacles. We here propose a perceptive shared control system that progressively corrects the trajectory as a user manually drives the wheelchair, by means of a sensor-based shared control law capable of smoothly avoiding obstacles. This control law is based on a low complex optimization framework validated through simulations and extensive clinical trials. The provided model uses distance information. Therefore, for low-cost considerations, we use ultrasonic sensors to measure the distances around the wheelchair. The solution therefore provides an efficient assistive tool that does not alter the quality of experience perceived by the user, while ensuring his security in hazardous situations.

An important aspect to be taken care of while designing assistive robots for mobility is that they need to operate among humans. Thus understanding human spatial social conventions and incorporating them in the assistive solutions, is important. In this paper, we introduce a semi-autonomous framework for assistive wheelchair navigation in human environments, which is driven by the intention of the wheelchair user. Safe and socially compliant motion provided by a user intention driven local motion planner is fused with user teleoperation in order to create such a system. Taking into account the fact that the user is the primary controller, our proposed system aims to provide progressive assistance whenever the user is in danger of collision or at risk of disturbance to other humans. We also thus propose generalized formulations for estimating user intentions and for sharing control within the context of wheelchair mobility assistance, that is adaptable in order to be deployed in real world systems. We then evaluate the proposed framework in simulation in order to obtain a quantitative analysis. We also provide experimental evidence using an off-the-shelf robotized wheelchair equipped with a single 2D laser scanner.

Since social, assistive and companion robots need to navigate within human crowds, understanding spatial social conventions while designing navigation solutions for such robots is an essential issue. This work presents analysis of an socially compliant robot motion strategy that could be employed by social robots such as humanoids, service robots or intelligent wheelchairs, for approaching and joining humans groups in interaction, and then become an equitable part of the interaction. Following our previous work that formalized the motion strategy, a detailed synthesis is presented here with experiments that validate the proposed system in the real world.

In this work we introduce a low-level system that could be employed by a social robot like a robotic wheelchair or a humanoid, for approaching a group of interacting humans, in order to become a part of the interaction. Taking into account an interaction space that is created when at least two humans interact, a meeting point can be calculated where the robot should reach in order to equitably share space among the interacting group. We propose a sensor-based control task which uses the position and orientation of the humans with respect to the sensor as inputs, to reach the said meeting point while respecting spatial social constraints. Trials in simulation demonstrate the convergence of the control task and its capability as a low-level system for human-aware navigation.

In case of motor impairments, steering a wheelchair can become a hazardous task. Typically, along corridors, joystick jerks induced by uncontrolled motions are source of wall collisions. This paper describes a vision based assistance solution for safe indoor semi-autonomous navigation purposes. To this aim, the control process is based on a visual servoing process designed for wall avoidance purposes. As the patient manually drives the wheelchair, a virtual guide is defined to progressively activate an automatic trajectory correction. The proposed solution does not require any knowledge of the environment. Experiments have been conducted over corridors that present different configurations and illumination conditions. Results demonstrate the ability of the system to smoothly and adaptively assist people during their motions.

When considering an indoor navigation without using any prior knowledge of the environment, relevant landmark extraction remains an open issue for robot localization and navigation. In this paper, we consider indoor navigation along corridors. In such environments, when considering monocular cameras, doors can be seen as important landmarks. In this context, we present a new framework for door detection and tracking which exploits geometrical features of corridors. Since real-time properties are required for navigation purposes, designing solutions with a low computational complexity remains a relevant issue. The proposed algorithm relies on visual features such as lines and vanishing points that are further combined to discriminate the floor and wall planes and then to recognize doors within the image sequences. Detected doors are used to initialize a dedicated edge-based 2D door tracker. Experiments show that the framework is able to detect 82% of doors on our dataset while respecting real time constraints.

In this paper, we present an autonomous navigation framework of a wheelchair by means of a single camera and visual servoing. We focus on a corridor following task where no prior knowledge of the environment is required. Our approach embeds an image-based controller, thus avoiding to estimate the pose of the wheelchair. The servoing process matches the non holonomous constraints of the wheelchair and relies on two visual features, namely the vanishing point location and the orientation of the median line formed by the straight lines related to the bottom of the walls. This overcomes the process initialization issue typically raised in the literature. The control scheme has been implemented onto a robotized wheelchair and results show that it can follow a corridor with an accuracy of ±3cm.

This paper presents an adaptive scheme for synthesizing missing textured regions. In synthesis-based compression approaches, large textures are removed at encoder side and filled in at decoder side. This work proposes a synthesizer in which both complementary pixel-based and patch-based approaches are used. According to results shown by synthesis algorithms, patch-based and pixel-based approaches are efficient with different kinds of texture. Two algorithms are adapted to the region synthesis context: the sample patch is built from surrounding texture and potential anchor blocks inside the removed region; the scan order depends on a confidence map in order to take advantage of the designed patch; DCT-descriptors provide the minimum size of matching window which is a required parameter for both synthesizers. Finally an "a posteriori" gradient based assessor enables the synthesizer to switch between algorithms.

Quality assessment is of major importance when designing and testing an image/video coding technique. Compression performances are usually evaluated by means of rate-distortion curves. However, the PSNR is commonly employed as the distortion measure. We hereby present a full quality assessment benchmark for the LAR (locally adaptive resolution) coder. We conducted a subjective experiment, where nineteen observers were asked to assess the perceptual quality of LAR coded images under normalized viewing conditions. Furthermore, five objective quality assessment metrics were used in order to determine the most suitable metric for the LAR coder. Finally, both JPEG and JPEG200 images were generated and assessed during the subjective experiment in order to define the optimal quality metric which should be used when comparing the codecs' output images quality.

In this paper, we introduce a novel way to represent an image sequence, which naturally exhibits the temporal persistence of the textures. Standardized representations have been thoroughly optimized, and getting significant improvements has become more and more difficult. As an alternative, Analysis-Synthesis (AS) coders have focused on the use of texture within a video coder. We introduce here a new AS representation of image sequences that remains close to the classic block-based representation. By tracking textures throughout the sequence, we propose to reconstruct it from a set of moving textures which we call motion tubes. A new motion model is then proposed, which allows for motion field continuities and discontinuities, by hybridizing block matching and a low-computational mesh-based representation. Finally, we propose a bi-predictional framework for motion tubes management.

Next generations of still image codecs should not only have to be efficient in terms of compression ratio, but also propose other functionalities such as scalability, lossy and lossless abilities, region of interest coding, etc. In previous works, we have proposed the LAR compression method covering these requirements. In particular, the RWHaT + P pyramid has recently been presented as a powerful reversible scalable coding technique. This paper introduces new significant improvements by the use of an inter-coefficient classification method. Results are discussed and compared to the state of the art.

Scalable Video Coding was designed in response to the growing need for flexibility in video transmission over networks and channels. MPEG-4 Scalable Video Coding (SVC) is a recently finalized standard which introduces new coding tools such as spatial, temporal and quality scalability, to produce a layer-based scalable video stream. Additionally, inter-layer prediction allows a layer to use information from other layers as a basis for motion and texture prediction, improving the overall coding efficiency. Rate control is a capital issue in video coding, as it is designed to regulate the bitrate at the output of the encoder and keep it close to a specified constraint. Whereas rate control has been extensively studied for non-scalable video coding, only few propositions were made for scalable video coding. In this paper, we adapt an attractive rate control approach, based on a bitrate modeling framework called $\rho$-domain, for scalable video coding. We show that this model performs well on all spatial, temporal and quality scalabilities, and handles inter-layer prediction quite accurately. After validating the approach in MPEG-4 SVC, we use the rho-domain model to build a simple-accurate rate control scheme. Results show that the mean frame bitrate error is below 7% on a representative set of configurations, while the impact on the complexity of the encoder is very low.

EROS is the largest database in the world of high resolution art pictures. The TSAR project is designed to open it in a secure, efficient and user-friendly way that involves cryptography and watermarking as well as compression and region-level representation abilities. This paper more particularly addresses the two last points. The LAR codec is first presented as a suitable solution for picture encoding with compression ranging from highly lossy to lossless. Then, we detail the concept of self-extracting region representation, which consists of performing a segmentation process at both the coder and decoder from a highly compressed image, and later locally enhancing the image in a region of interest. The overall scheme provides an efficient, consistent solution for advanced data browsing.

We present an efficient content-based image coding called locally adaptive resolution (LAR) offering advanced scalability at different semantic levels, i.e., pixel, block, and region. A local analysis of image activity leads to a nonuniform block representation supporting two layers of image description. The first layer provides global information encoded in the spatial domain enabling a low bit rate while preserving contours. The second layer holds texture information encoded in the spectral domain, enabling scalable bitstream in accordance with the required quality. This basic LAR coding leads to an efficient progressive compression, evaluated through subjective quality tests. Its nonuniform block representation also allows a hierarchical region representation providing higher semantic functionalities. More precisely, the segmentation process can be simultaneously performed at both the coder and the decoder from only the luminance component highly compressed by the first coding layer. This solution provides a representation at a region level while avoiding any contour encoding overhead. Region enhancement can then be realized through the second layer. Furthermore, very high compression of the chromatic components is achieved thanks to this region representation. In this scheme, a low-cost chromatic control, which was first introduced during the segmentation process, increases the consistency of region representation in terms of color.

Scalability and others functionalities such as the region of interest encoding become essential properties of an efficient image coding scheme. Within the framework of lossless compression techniques, S+P and CALIC represent the state-of-the-art. The proposed interleaved S+P algorithm outperforms these method while providing the desired properties. Based on the LAR (locally adaptive resolution) method, an original pyramidal decomposition combined with a DPCM scheme is elaborated. This solution uses the S-transform in such a manner that a refined prediction context is available for each estimation steps. The image coding is done in two main steps, so that the first one supplies a LAR low-resolution image of good visual quality, and the second one allows a lossless reconstruction. The method exploits an implicit context modelling, intrinsic property of our content-based quad-tree like representation.

We present a new scalable compression technique dealing simultaneously with both lossy and lossless image coding. An original DPCM scheme with refined context is introduced through a pyramidal decomposition adapted to the LAR (locally adaptive resolution) method, which becomes by this way fully progressive. An implicit context modeling of the prediction errors, due to the low resolution image representation including variable block size structure, is then exploited to the for lossless compression purpose.

We present a new scalable compression technique dealing simultaneously with both lossy and lossless image coding. An original DPCM scheme with refined context is introduced through a pyramidal decomposition adapted to the LAR (locally adaptive resolution) method, which becomes by this way fully progressive. An implicit context modeling of the prediction errors, due to the low resolution image representation including variable block size structure, is then exploited to the for lossless compression purpose.