This study addresses the critical need for effective rehabilitation monitoring for individuals with gait issues through the integration of interpretable artificial intelligence (AI) models. The study meticulously details the methodology, emphasizing the systematic data collection process, participant categorization based on gait severity, and the deployment of a novel physical activity monitoring device. Through the lens of the Berg balance test, the analysis scrutinizes functional mobility and balance across seven rehabilitation activities. Data augmentation techniques and data preprocessing methods are employed, and the proposed hybrid model combines both time and frequency-domain data using a multibranch CNN-BiLSTM architecture embedded with a dual-stage attention mechanism. The study compares our approach with machine learning and deep learning models, showcasing the superior performance of the proposed interpretable architecture. Interpretability is highlighted through attention mechanisms, revealing the model’s prioritization of fast Fourier transform-transformed data and its ability to identify influential branches. Our model provides transparent insights to healthcare professionals for personalized rehabilitation strategies. These results show high potential in developing real-time applications, including the integration of additional sensor modalities, interdisciplinary collaborations, and the expansion of demographic inclusivity for more robust and applicable rehabilitation monitoring solutions.

Traditional methods for quality inspection of reinforcing bars (rebars) in reinforced concrete (RC) structures are labor-intensive, time-consuming, and pose safety risks. Recent methods utilizing advanced sensor technologies, despite their effectiveness, are often too costly, limiting their widespread use. Additionally, vision-based alternatives often sacrifice accuracy in measurements and geometry. In response to these challenges, this paper introduces a robust framework that uses point cloud data captured by iPhone LiDAR sensor for the efficient and cost-effective rebar inspection. Our approach is articulated through two principal modules: the Component Recognition Module (CRM) and the Dimensional Assessment Module (DAM). The CRM incorporates an adapted RandLA-Net model for the semantic segmentation of point clouds, enabling the precise identification of rebar components. Meanwhile, the DAM employs an adapted RANSAC algorithm for evaluating rebar spacing. Based on our framework's evaluation, we achieved an overall accuracy of 82.1% in the semantic segmentation task and a mean absolute error of 0.9 cm in the rebar spacing evaluation task. The results demonstrate that our proposed framework provides accurate rebar inspection while remaining cost-effective.

The adoption of deep learning in healthcare is hindered by their “black box” nature. In this paper, we explore the RETAIN architecture for the task of glucose forecasting for diabetic people. By using a two-level attention mechanism, the recurrent-neural-network-based RETAIN model is interpretable. We evaluate the RETAIN model on the type-2 IDIAB and the type-1 OhioT1DM datasets by comparing its statistical and clinical performances against two deep models and three models based on decision trees. We show that the RETAIN model offers a very good compromise between accuracy and interpretability, being almost as accurate as the LSTM and FCN models while remaining interpretable. We show the usefulness of its interpretable nature by analyzing the contribution of each variable to the final prediction. It revealed that signal values older than 1[Formula: see text]h are not used by the RETAIN model for 30[Formula: see text]min ahead of time prediction of glucose. Also, we show how the RETAIN model changes its behavior upon the arrival of an event such as carbohydrate intakes or insulin infusions. In particular, it showed that the patient’s state before the event is particularly important for the prediction. Overall the RETAIN model, thanks to its interpretability, seems to be a very promising model for regression or classification tasks in healthcare.

We present an observational study with domain experts to understand how augmented reality (AR) extensions to traditional PC-based data analysis tools can help particle physicists to explore and understand 3D data. Our goal is to allow researchers to integrate stereoscopic AR-based visual representations and interaction techniques into their tools, and thus ultimately to increase the adoption of modern immersive analytics techniques in existing data analysis workflows. We use Microsoft's HoloLens as a lightweight and easily maintainable AR headset and replicate existing visualization and interaction capabilities on both the PC and the AR view. We treat the AR headset as a second yet stereoscopic screen, allowing researchers to study their data in a connected multi-view manner. Our results indicate that our collaborating physicists appreciate a hybrid data exploration setup with an interactive AR extension to improve their understanding of particle collision events.

Previous studies about kinesthetic expressions of emotions are mainly based on acted expressions of emotions, which can be quite different from spontaneous expressions. This paper describes a study involving a stress induction procedure and stress perception through haptic expressions with Ntotal= 41 young men (aged 19 - 37), all right-handed. We designed a game application to collect spontaneous expressions of stress. This application involved haptic interactions and a stressful event. We observed changes in the haptic behaviors of the participants over different phases: before, during and after the stress induction. In the next step, we investigated how the collected haptic behaviors (both kinematic and force components) were haptically perceived by the other participants. The results suggest the ability of kinesthetic expressions to communicate a spontaneous stress from one person to another.

This paper presents the Derivatives Combination Predictor (DCP), a novel model fusion algorithm for making long-term glucose predictions for diabetic people. First, using the history of glucose predictions made by several models, the future glucose variation at a given horizon is predicted. Then, by accumulating the past predicted variations starting from a known glucose value, the fused glucose prediction is computed. A new loss function is introduced to make the DCP model learn to react faster to changes in glucose variations. The algorithm has been tested on 10 in-silico type-1 diabetic children from the T1DMS software. Three initial predictors have been used: a Gaussian process regressor, a feed-forward neural network and an extreme learning machine model. The DCP and two other fusion algorithms have been evaluated at a prediction horizon of 120 minutes with the root-mean-squared error of the prediction, the root-mean-squared error of the predicted variation, and the continuous glucose-error grid analysis. By making a successful trade-off between prediction accuracy and predicted-variation accuracy, the DCP, alongside with its specifically designed loss function, improves the clinical acceptability of the predictions, and therefore the safety of the model for diabetic people.

Research in diabetes, especially when it comes to building data-driven models to forecast future glucose values, is hindered by the sensitive nature of the data. Because researchers do not share the same data between studies, progress is hard to assess. This paper aims at comparing the most promising algorithms in the field, namely Feedforward Neural Networks (FFNN), Long Short-Term Memory (LSTM) Recurrent Neural Networks, Extreme Learning Machines (ELM), Support Vector Regression (SVR) and Gaussian Processes (GP). They are personalized and trained on a population of 10 virtual children from the Type 1 Diabetes Metabolic Simulator software to predict future glucose values at a prediction horizon of 30 minutes. The performances of the models are evaluated using the Root Mean Squared Error (RMSE) and the Continuous Glucose-Error Grid Analysis (CG-EGA). While most of the models end up having low RMSE, the GP model with a Dot-Product kernel (GP-DP), a novel usage in the context of glucose prediction, has the lowest. Despite having good RMSE values, we show that the models do not necessarily exhibit a good clinical acceptability, measured by the CG-EGA. Only the LSTM, SVR and GP-DP models have overall acceptable results, each of them performing best in one of the glycemia regions.

Abstract We discuss spatial selection techniques for three‐dimensional datasets. Such 3D spatial selection is fundamental to exploratory data analysis. While 2D selection is efficient for datasets with explicit shapes and structures, it is less efficient for data without such properties. We first propose a new taxonomy of 3D selection techniques, focusing on the amount of control the user has to define the selection volume. We then describe the 3D spatial selection technique Tangible Brush, which gives manual control over the final selection volume. It combines 2D touch with 6‐DOF 3D tangible input to allow users to perform 3D selections in volumetric data. We use touch input to draw a 2D lasso, extruding it to a 3D selection volume based on the motion of a tangible, spatially‐aware tablet. We describe our approach and present its quantitative and qualitative comparison to state‐of‐the‐art structure‐dependent selection. Our results show that, in addition to being dataset‐independent, Tangible Brush is more accurate than existing dataset‐dependent techniques, thus providing a trade‐off between precision and effort.

Abstract We present a pressure‐augmented tactile 3D data navigation technique, specifically designed for small devices, motivated by the need to support the interactive visualization beyond traditional workstations. While touch input has been studied extensively on large screens, current techniques do not scale to small and portable devices. We use phone‐based pressure sensing with a binary mapping to separate interaction degrees of freedom (DOF) and thus allow users to easily select different manipulation schemes (e. g., users first perform only rotation and then with a simple pressure input to switch to translation). We compare our technique to traditional 3D‐RST (rotation, scaling, translation) using a docking task in a controlled experiment. The results show that our technique increases the accuracy of interaction, with limited impact on speed. We discuss the implications for 3D interaction design and verify that our results extend to older devices with pseudo pressure and are valid in realistic phone usage scenarios.

With the development of compliant mechanisms and tactile skins, social robots are more inclined to physically interact with humans. Touch has been proved highly beneficial in interpersonal social interaction. This motivates social robots to exhibit affective touch abilities. However, most of the studies tackle the consequences and the perception of touch instead of how touching is performed. Understanding the touch manner enables to control robots in a more rich and natural way, but also to detect the intention of the user while touching the robot. This paper focuses on one specific social interaction based on touch, the handshake, and it analyses the pressure exerted by the participants depending on their mood. A corpus of daily repeated handshakes of 11 participants, during 16 days in Human-Human and Human-Robot interactions was produced. A handshake descriptor of 6 variables was computed and allowed to discriminate the participants' behavior. One of the variables (i.e., the maximum pressure on the fingers) enabled to discriminate the mood “Bored” from the others.

In this article, effectiveness of air jet stimulation in mediated emotional communication was investigated by assessing cross-modal influences of visual emotional expressions on tactile perception. Brain responses to combined visual faces and air jet stimuli were measured using event-related potentials; whereas, emotional responses were assessed using self-reported pleasantness of the tactile stimulation. ERP results reveal significant differences between the different facial expressions for the same tactile air-jet intensity in the somatosensory area. Moreover, participants’ pleasantness ratings suggest an effect of the visual stimulus on the difference tactile conditions that correspond to air jet stimulation intensities: low, medium, and high. These promising results provide evidence in the potential efficiency of this stimulation technique in activating skin receptors that play an important role in social and affective behaviors.

We conducted an experimental study to investigate how the participants combine touch and facial expression to evaluate emotional valence. In this study, visual and haptic stimuli were presented separately and then presented together. The visual stimuli comprised pictures of facial expressions with different emotional levels, and the touch stimuli consisted of air jet tactile stimulation performed on the arms of the participants. The participants were asked to evaluate the communicated emotional valence on a continuous scale. Information Integration Theory was used to model the algebraic rule (addition, multiplication, averaging, and so on) that underlies the multimodal perception of emotional valence. Analyses showed that the participants usually integrated both visual and touch information to evaluate the emotional valence. The main integration rule was averaging, with nonsystematic predominance of each modality over the other modality.

We present the design and evaluation of pressure-based interactive control of 3D navigation precision. Specifically, we examine the control of gain factors in tangible 3D interactions using locally-coupled mobile devices. By focusing on pressure as a separate input channel we can adjust gain factors independently from other input modalities used in 3D navigation, in particular for the exploration of 3D visualisations. We present two experiments. First, we determined that people strongly preferred higher pressures to be mapped to higher gain factors. Using this mapping, we compared pressure with rate control, velocity control, and slider-based control in a second study. Our results show that pressure-based gain control allows people to be more precise in the same amount of time compared to established input modalities. Pressure-based control was also clearly preferred by our participants. In summary, we demonstrate that pressure facilitates effective and efficient precision control for mobile 3D navigation.

We evaluate the performance and usability of mouse-based, touch-based, and tangible interaction for manipulating objects in a 3D virtual environment. This comparison is a step toward a better understanding of the limitations and benefits of these existing interaction techniques, with the ultimate goal of facilitating an easy transition between the different 3D data exploration environments. For this purpose we analyze participants' performance in 3D manipulation using a docking task. We measured completion times, docking accuracy, as well as subjective criteria such as fatigue, workload, and preference. Our results show that the three input modalities provide similar levels of precision but require different completion times. We also discuss our qualitative observations as well as people's preferences and put our findings into context of the application domain of 3D data analysis environments.

In this paper, we propose to determine whether the viewer's behavior changes or not before, during and after watching a TV program. Are there any behaviors specific to each particular phase of viewing? Here, we propose a flexible and nonintrusive method based on the use of three categories of everyday connected objects (i.e. Smartphone, smartwatch and remote control). Data were collected during participants' interactions with a smart TV i.e. during three phases of viewing a TV program: before, during and after in an uncontrolled environment. A classification of these behaviors was done in order to identify the behaviors of viewers during each phase of TV-watching using a Deep Neural Network (DNN) algorithm, without prior pre-processing of the learning data. The results show that the viewers' behavior is very different from each other during the three phases. Thus, the behavior that was detected in two different phases did not have the same frequency of occurrence for all participants and was not performed by a given viewer with the same duration. In addition, the DNN algorithm gives a good average recognition performance of the three phases: 92.75%, 87.5% and 94% for the first second and third phase respectively. This study can help to better understand the viewers' behavior and anticipate certain actions in order to maintain their attention towards the proposed TV programs.

International audience

We present the design and evaluation of an interface that combines tactile and tangible paradigms for 3D visualization. While studies have demonstrated that both tactile and tangible input can be efficient for a subset of 3D manipulation tasks, we reflect here on the possibility to combine the two complementary input types. Based on a field study and follow-up interviews, we present a conceptual framework of the use of these different interaction modalities for visualization both separately and combined-focusing on free exploration as well as precise control. We present a prototypical application of a subset of these combined mappings for fluid dynamics data visualization using a portable, position-aware device which offers both tactile input and tangible sensing. We evaluate our approach with domain experts and report on their qualitative feedback.

In this work, we present a platform for continuously monitor and guide stroke patient at home during Activities of the Daily Living. The platform consists of a smart cup which embeds sensors that monitor the patient's hand and arm motor activity at different times of the day. The cup detects its orientation, the liquid level, its position on a specific target, as well as tremors. Moreover, displays are provided to guide the patient's movement. Finally, the planned studies with both the therapists and patients are presented.

Recognition of human activity in a non-controlled environment remained an unsolved problem. In this study, we wonder if accurate recognition of activity can be obtained using Internet of Things technology. We propose a processing methodology that can allow to an automatic system like a robot to recognize human activity. The approach consists of the classification of some activities of the subjects: walking, standing, sitting and laying. The study exploits a standard smartwatch and a smartphone carried by participants during a non-controlled experiment. We propose a pre-computation using Discrete Cosine Transform (DCT), and we identify the best window width and feature length that provides the best results. We show that Support Vector Machines (SVM) provides better results compared with Decision Tree algorithms (DT). The results also demonstrate that participants' activities were classified with an accuracy of more than 91% in a non-controlled environment, with a non-controlled position of the smartphone. We define the notion of transient that corresponds to the transition between two activities as well between two positions of the sensors. The last result shows that removing the transients provides better results for the classification, i.e. 98%. This approach shows that it is possible for a robot assistant to understand the human behavior only using standard Internet of Things technology in a non-controlled environment.

This paper presents a preliminary study which aims to investigate the tangible navigation in 3D Virtual Environments with new tablets (ex. Google Tango) that provide a self-contained 3D full tracking (ex. Google Tango, Intel RealSense). The tangible navigation was compared with tactile navigation techniques used in standard 3D applications (ex. video games, CAD, data visualization). Four conditions were compared: classic multi-touch interaction, tactile interaction with sticks, tangible interaction with a 1:2.5 scale factor and tangible interaction with a 1:5 scale factor. The study focuses on the subjective evaluation users make of the different interaction techniques in terms of usability and acceptability, and the fidelity of representation of the explored scene. Participants were asked to explore a virtual appartment using one of the four interaction techniques. Then, they were requested to draw a 2D sketch map of the explored appartment and to complete a questionnaire of usability and acceptabilty. In order to go further into the investigation, the participant's activity has also been recorded. First results show that exploring a virtual environment using only the device's movements is more usable and acceptable compared to tactile interaction technique, but the difference seems to become smaller as the used scale factor increases.

This paper presents a mobile platform that uses a new generation of tablets equipped with a depth sensor to perform a real time 3D reconstruction of an indoor environment. The platform generates a 3D model where the strucutral elements are identified: ground, ceiling, walls and openings. The 3D model is used for the evaluation of both the geometric features of the building, but also for the assessment of the building's energetic performance. Also, a series of edition and visualization tools are proposed within the platform to assist the user in modifying and exploring the 3D reconstructed environment.

Previous studies about kinesthetic expressions of emotions are mainly based on acted expressions of affective states, which might be quite different from spontaneous expressions. In a previous study, we proposed a task to collect haptic expressions of a spontaneous stress. In this paper, we explore the effectiveness of this task to induce a spontaneous stress in two ways: a subjective feedback, and a more objective approach-avoidance behavior.

The use of virtual avatars, through facial or gestural expressions, is considered to be a main support for affective communication. Currently, different works have studied the potential of a kinesthetic channel for conveying such information. However, they still have not investigated the complementarity between visual and kinesthetic feedback to effectively convey emotion. This paper studies the relation between some emotional dimensions and the visual and kinesthetic modalities. The experimental results show that subjects used visual and kinesthetic feedbacks to evaluate the pleasure and the arousal dimensions, respectively. We also observed a link between the recognition rate of emotions expressed with the visual modality (resp. kinesthetic modality) and the magnitude of that emotion's pleasure dimension (resp. arousal dimension). These different results should help in the selection of feedbacks according to the features of the investigated emotion.

This study presents an experiment highlighting how participants combine facial expressions and haptic feedback to perceive emotions when interacting with an expressive humanoid robot. Participants were asked to interact with the humanoid robot through a handshake behavior while looking at its facial expressions. Experimental data were examined within the information integration theory framework. Results revealed that participants combined Facial and Haptic cues additively to evaluate the Valence, Arousal, and Dominance dimensions. The relative importance of each modality was different across the emotional dimensions. Participants gave more importance to facial expressions when evaluating Valence. They gave more importance to haptic feedback when evaluating Arousal and Dominance.

This paper deals with the haptic affective social interaction during a greeting handshaking between a human and a humanoid robot. The goal of this work is to study how the haptic interaction conveys emotions, and more precisely, how it in'uences the perception of the dimensions of emotions expressed through the facial expressions of the robot. Moreover, we examine the bene'ts of the multimodality (i.e., visuo-haptic) over the monomodality (i.e., visual-only and haptic-only). The experimental results with Meka robot show that the multimodal condition presenting high values for grasping force and joint stiffness are evaluated with higher values for the arousal and dominance dimensions than during the visual condition. Furthermore, the results corresponding to the monomodal haptic condition showed that participants discriminate well the dominance and the arousal dimensions of the haptic behaviours presenting low and high values for grasping force and joint stiffness.

Coordinating team's work in a 3D Collaborative Virtual Environments is a real challenge. The communication constraints in such environments reduces the understanding between the partner's and thus complicates the team's coordintation. In order to facilitate the coordination, this paper proposes a hap tic tool for the designation of targets. This tool was experimented in a context of collaborative molecular design where one user designates tasks and coordinates the actions of two operators. The experimental results show an improvement of the working efficiency of the group and a better sharing of the activity between partners. Moreover, we observe that the hap tic guidance enables an accurate selection of targets which leads to an improvement of performance during the deformation process.

In mixed reality, real objects can be used to interact with virtual objects. However, unlike in the real world, real objects do not encounter any opposite reaction force when pushing against virtual objects. The lack of reaction force during manipulation prevents users from perceiving the mass of virtual objects. Although this could be addressed by equipping real objects with force-feedback devices, such a solution remains complex and impractical. In this work, we present a technique to produce an illusion of mass without any active force-feedback mechanism. This is achieved by simulating the effects of this reaction force in a purely visual way. A first study demonstrates that our technique indeed allows users to differentiate light virtual objects from heavy virtual objects. In addition, it shows that the illusion is immediately effective, with no prior training. In a second study, we measure the lowest mass difference (JND) that can be perceived with this technique. The effectiveness and ease of implementation of our solution provides an opportunity to enhance mixed reality interaction at no additional cost.

Haptic expression of emotions has received less attention than other modalities. Bonnet et al. [2011] combine visio-haptic modalities to improve the recognition and discrimination of some emotions. However, few works investigated how these modalities complement each other. For instance, Bickmore et al. [2010] highlight some non-significant tendencies of complementarity between the visual and haptic modalities.

Designating targets, such as elementary primitives (e.g., vertices, edges and faces) or complex objects (e.g., 3D objects and structures), to a partner in Collaborative Virtual Environments is a real challenge in different applications. In fact, the communication constraints in such environments limit the understanding of the partner's actions, which lead to wrong selections and thus to conflicting actions. Beyond these limitations, applications providing complex data to manipulate, such as molecular environments, introduce additional constraints which limit the perception and access to the partner's working space. This paper proposes a remote designation procedure linked with an haptic attraction model for molecular deformation tasks, we propose to guide physically the partner to the designated atom. Experimental results showed a significant improvement in performance and efficiency for the different steps of collaborative tasks (i.e., designation/selection of atoms and deformation of structures). Moreover, this haptic guidance method enables more accurate selections of atoms for the partner.

Haptic devices are dedicated to render virtual tactile stimulation. A limitation of these devices is the intrusiveness of their mechanical structures, i.e. the user need to hold or wear the device to interact with the environment. Here, we propose a concept of new tactile device named HAIR. The device is composed of a computer vision system, a mechatronic device and air jets that stimulate the skin. We designed a first prototype and conducted a preliminary experiment to validate our concept. The interface enables a tactile interaction without using physical contact with material devices, providing better freedom of movement and enhancing the interaction transparency.

Exploration of volumetric data is an essential task in many scientific fields. However, the use of standard devices, such as the 2D mouse, leads to suboptimal interaction mappings. Several VR systems provide better interaction capabilities, but they remain dedicated and expensive solutions. In this work, we propose an interface that combines tangible tools and a handheld device. This configuration allows natural and full 6 DOF interaction in a convenient, fully portable and affordable system. This paper presents our design choices for this interface and associated tangible exploration techniques.

Applications requiring processing of complex environments present real challenges for simultaneous management of numerous constraints (multiple degrees of freedom, search criterion, etc.). Target searching is probably among the most critical tasks, consisting of finding configurations corresponding to various criteria (e.g., maximum, minimum, reference). During the search, users need to be aware of proximal results to compare values and make decisions. A new audio-haptic coupling strategy is proposed to improve the search for targets in complex environments, enabling the simultaneous use of both audio and haptic channels for value comparisons at different spatial configurations. This is accomplished through the use of tempo in both sensorial signals creating a connection between the two channels enabling an intuitive and efficient comparison. Including spatialized audio improves a user’s situation awareness. The benefit of this intermodal metaphor is evaluated for a 2D nonvisual and abstract environment. Results show improvements relative to simple haptic exploration.

Several studies have investigated the relevance of haptics to convey various types of emotions physically. This article investigates the improvement of the recognition rate of emotions using visuo-haptic feedback compared to facial and haptic expressions alone. Four experiments were conducted in which the recognition rates of emotions using facial, haptic and visuo-haptic expressions were tested. The first experiment evaluates the recognition rate of emotions using facial expressions. The second experiment collects a large corpus of 3D haptic expressions of certain emotions and subsequently identifies the relevant haptic expression for each emotion. The third experiment evaluates the selected haptic expressions through statistical and perceptive tests to retain the ones that result in the most accurate identification of the corresponding emotion. Finally, the fourth experiment studies the effect of visuo–haptic coupling on the recognition of the investigated emotions. Generally, emotions with high amplitudes of pleasure are better recognized in the visual modality. However, emotions with high activation are better recognized in the haptic modality. These results also highlighted the finding that participants are not equally aided by each modality when recognizing emotions efficiently. Beyond the recognition rate, multimodal expressions improved the sensation of presence and expressivity.

Eliciting emotions to collect affective expressions is frequently used in studies about facial, spoken or bodily expressions of emotions. However, recent works in affective hap tics focus mostly on expressions of acted emotions. The expression of spontaneous emotions using the haptic modality remains unexplored. This paper discusses methodological considerations for designing experimental protocols to collect haptic expressions of spontaneous emotions. We highlight the main difficulties to solve, provide illustrative examples from several experiments that we conducted, and describe methodological considerations for eliciting emotions and collecting corresponding haptic expressions.

Several studies in affective computing consider the role of haptic channel via the tactile and kinesthetic components. However, these works used constraining haptic devices with limited stimulation strategies. This paper proposes to investigate a new haptic device for the tactile stimulation with a mobile air jet. This approach is more suitable for the stimulation of skin mechanoreceptors which play an important role in social and affective behaviors. An experimental study was conducted in order to understand the relationships between the physical features of the air jet and the perception of emotions by participants. The results highlight a main effect of the intensity and movement velocity of the air jet stimulation on the evaluations of valence, arousal, and dominance.

Haptic rendering technologies are becoming a strategic component of the new Human-Machines Interfaces. Many existing devices generally operate with intrusive mechanical structures that limit rendering and transparency of the haptic interaction. In the context of the design of a novel non-contact haptic device that exploit the air jet tactile stimulation with long distance, we carried out psychophysical experiments in order to characterize the human perception of the air jet tactile stimulation. This study focuses on the stimulation of the user's hand palm. Fluid mechanics suggests that perceived parameters of the air jet stimulation can be controlled by the air jet flow rate, the nozzle geometry, and the distance between the nozzle and the hand palm. In this paper we investigate how these control variables affect the perceived force on the hand palm. Two psychophysical studies were carried out. The first experiment investigated three distances from the air nozzle to determine the absolute threshold of the air flow rate on the hand palm. Results reveal that there is a linear relationship between the perceived absolute threshold and the distance from the hand palm to the nozzle. The second psychophysical experiment was made to determinate the just noticeable difference of the air flow rate according to three referential stimuli. Then we estimated the Weber fraction. These two results are important for the display control and will be used further for the design of the future noncontact haptic device.

Exploring virtual environment with immersive metaphor is still largely unexplored, with the costly CAVE exception. This question takes importance in lots of fields, such as fluid mechanics, where space and time resolved dataset become more and more common. For that reason, we present an interaction design study of an window exploration metaphor, for large 3D virtual environment. The metaphor is based on the use of a tablet as a tangible and movable window on a virtual environment. Rotations in the environment are tracker-less mapped on the rotations of the tablet. Our design is inspired by fluid mechanics issues, but is build keeping generalizability in mind. The study shows that mapping three degrees of freedom onto corresponding real three degrees of freedom of space raises the transparency, the efficiency of data exploration and the space awareness of users.

Several studies have investigated the relevance of haptics to physically convey various types of emotion. However, they use basic analysis approaches to identify the relevant features for an effective communication of emotion. This article presents an advanced analysis approach, based on the clustering technique, that enables the extraction of the general features of affective haptic expressions as well as the identification of specific features in order to discriminate between close emotions that are difficult to differentiate. This approach was tested in the context of affective communication through a virtual handshake. It uses a haptic device, which enables the expression of 3D movements. The results of this research were compared to those of the standard Analysis of Variance method in order to highlight the advantages and limitations of each approach.

Several studies showed the relevance of haptics for conveying emotions to users. These studies usually cover recognition rate of emotions from haptic expressions. Surprisingly, the analysis of features of these haptic expressions is in counterpart often limited to a classical analysis of variance. However, this method suffers of several limitations for the analysis of such data. This paper investigates the complementarity of three different statistical methods for the processing of haptic expressions of emotions.

Mutual awareness between users working in collaborative virtual environments is an important factor for efficient collaboration. This article presents a collaborative metaphor dedicated to the dynamic adjustment of selection area in polygonal modelling.

Collaborative virtual environments introduce new working methods allowing for the association of several experts in the same problem-solving process. These new platforms have the potential to improve the processing of complex environments with large amounts of data and require different skills. This paper proposes the study of a synchronous and colocated approach for molecular design tasks. More precisely, this study focuses on the collaborative search of targets during the molecular deformation process. The aim of this paper is to highlight the different working strategies that emerge according to different tasks and humans factors. Based on these working strategies, we propose to investigate the contribution of collaborative configurations of work according to different efficiency criteria such as the completion time, verbal communication, and manipulation distance. Finally, this study will highlight some issues related to conflict of actions, concurrent access, awareness, and communication processes.

This article proposes to study the role of Collaborative Virtual Environments for the search of residues in molecular environments. This research highlights involved working strategies according the type and context of the task and shows some constraints and conflicting actions that may occur during closely coupled collaboration.

This study investigated how people achieve pointing tasks using a vibrotactile device. A comparison using visual feedback when doing the same task is presented. Subjects were asked to reach a target as fast and as precise as possible guided by visual information from an LCD screen; and two types of tactile information generated by a specific glove we developed. The first type of tactile information was a constant vibrating stimulation and the second one was a frequency modulation based vibration. For each modality, two experiments were performed. The first one was in 1-D: reaching a target situated on a horizontal left-right line. The second one was in 2-D: reaching a target is in a horizontal plane. The results we obtained showed that tactile based pointing task obey to the classical the Fitts' law. Furthermore, results showed that frequency modulation based vibrating gave better performance in trajectory length, but not in time of execution.

In the context of virtualized-reality-based telemicromanipulation, this paper presents a visual calibration technique for an optical microscope coupled to a charge-coupled device (CCD) camera. The accuracy and flexibility of the proposed automatic virtual calibration method, based on parallel single-plane properties, are outlined. In contrast to standard approaches, a 3-D virtual calibration pattern is constructed using the micromanipulator tip with subpixel-order localization in the image frame. The proposed procedure leads to a linear system whose solution provides directly both the intrinsic and extrinsic parameters of the geometrical model. Computer simulations and real data have been used to test the proposed technique, and promising results have been obtained. Based on the proposed calibration techniques, a 3-D virtual microenvironment of the workspace is reconstructed through the real-time imaging of two perpendicular optical microscopes. Our method provides a flexible, easy-to-use technical alternative to the classical techniques used in micromanipulation systems.

Our goal is to investigate new interaction techniques, using the haptic modality for CAD applications. An important issue in CAD systems is the modification of the Boundary Representation (B-Rep) of 3D objects. However, this fundamental task is only possible if a geometric element (vertex, edge, face) has already been selected. This paper focuses on several haptic selection methods to solve specific geometric constraints (differentiation of topological entities, variable density of the geometry, concavities) during the edition of a CAD model. Evaluation studies are carried out to compare the impact of these haptic methods on user performance. The results of these experiments are used to create a global haptic selection method for CAD editing.

In this paper we investigate attraction techniques, using the haptic modality, for CAD applications. An important issue in CAD systems is the modification of the Boundary Representation (B-Rep) of 3D objects. However, this fundamental task is possible only if a geometric element (vertex, edge, face) has already been selected. This paper focuses on a generic force feedback model of haptic attraction for CAD applications. Our model is flexible and allows the experimentation of several behaviours. Several evaluation studies are carried out to compare the impact of the model parameters on the user performance and comfort of use.

In this paper we present a comparative evaluation of haptic renderings for CFD data studied in virtual reality environments. We implemented haptics on both explicit (polygonal) and volumetric representations of isosurfaces. The haptic methods implemented go from attractive renderings, to volumic transfer functions as well as rigid contact methods. We lead several experiments to find the optimal haptic representations for CFD data analysis.

This paper describes the use of virtual fixtures and metaphors of assistance for robotic-assisted micromanipulation system in order to prevent the influence of microphysics on path planning and handling tasks. The system is based on a multimodal telemanipulation system using haptic/visual/sound interfaces for observation of microobjects under an optical microscope. Feasible haptically-generated paths based on potentials fields reaction forces and shock absorbers are described for efficient and safe pushing-based or adhesion-based micromanipulation. Then, metaphors with human sensory substitution are proposed in order to improve the perception of data or events. Finally, an experimental investigation carried out by nine trainees proves that the system guides efficiently and safely the operator's gesture. Moreover, user performance on a given task can increase as much as 52% in typical micromanipulation tasks.

This paper presents a new three-dimensional biomicromanipulation system for biological objects such as embryos, cells or oocytes. As the cell is very small, kept in the liquid, and observed through a microscope, the two-dimensional visual feedback makes difficult accurate manipulation in the 3-D world. To improve the manipulation work, we proposed an augmented human-machine interface. A 3-D visual information is provided to the operator through a 3-D reconstruction method using vision-based tracking deformations of the cell embryo. In order to stable injection tasks, the operator needs force feedback and haptic assistance during penetrating the cell envelop, the chorion. The proposed human-machine user's interface allows real-time realistic visual and haptic control strategies for constrained motion in image coordinates. Virtual haptic rendering allows to constrain the path of insertion and removal in the 3-D scene or to avoid cell destruction by controlling adequately position, velocity and force parameters

This paper presents a new bio-micromanipulation system for biological object such as embryo, cell or oocyte. As the cell is very small, kept in the liquid, and observed through a microscope, the two-dimensional visual feedback makes difficult accurate manipulation in the 3-D world. To improve the manipulation work, we proposed an augmented human-machine interface. A 3-D visual information is provided to the operator through a 3-D reconstruction method using vision-based tracking deformations of the cell embryo. In order to stable injection tasks, the operator needs force feedback and haptic assistance during penetrating the cell envelop, the chorion. The proposed human-machine user’s interface allows real-time realistic visual and haptic control strategies for constrained motion in image coordinates. Virtual haptic rendering allows to constrain the path of insertion and removal in the 3-D scene or to avoid cell destruction by controlling adequately position, velocity and force parameters.

In the context of virtualized reality based telemicromanipulation, we present in this paper a visual calibration technique for optical microscope coupled with a CCD camera. In contrast to previous approaches, a virtual calibration pattern is constructed using the micromanipulator with a sub-pixel localization in the image. We also present a new camera calibration algorithm based on Parallel Single-Plane properties. The proposed procedure leads to a linear system from which the solution gives directly both intrinsic and extrinsic parameters of the geometrical model. Both computer simulation and real data have been used to test the proposed technique, and very good results have been obtained. Compared with classical techniques, our method provides an alternative technical solution, easy to use and flexible in the context of micromanipulation and virtual reality.

Operators suffer much difficulty in manipulating micro/nano-sized objects without the assistance of human-machine interfaces due to scaling effects. We developed an immersive telemanipulation system using haptic/visual/sound interfaces for observation of micro-objects under an optical microscope. As the image of the microscope is two-dimensional, so it is hard to observe the workspace in the 3-D space. To improve the real-time observation and manipulation, we proposed real-time 3-D reconstruction of the microworld using image processing and virtualized reality techniques. Then, feasible haptically-generated paths based on potentials fields reaction forces are selected for efficient pushing-based manipulation without collisions. The proposed system guides the operator's gesture fully immerged in the virtual workspace.

This paper proposes a practical planning system for 2D assembly tasks at the microscale. To improve the real-time observation and manipulation, we proposed real-time 3-D reconstruction of the microworld using image processing and virtualized reality techniques. Then, a planner covers a whole range of problems in object assignment, obstacle detection and avoidance and path trajectory. We describe algorithms based on optimization theory and Voronoi graph construction. As automatic motion planners fail due to the difficulty of discovering critical configurations due to microdomain interactions, cooperation with operator skills is able to solve a motion planning query. The feasible haptically-generated paths based on potentials fields reaction forces are selected for efficient pushing-based manipulation without collisions.