Impact force is common haptic feedback on virtual reality (VR) controllers, such as hitting objects with weapons or rackets. It applies to different points of application of force (PAFs) and directions in varied scenarios. For example, using weapons with different shapes, e.g., a sword and a pickaxe, using a weapon in different ways, e.g., stabbing and slashing with a sword, or a ball flying and hitting a racket in different directions cause different PAFs and/or force directions. Therefore, to achieve realistic VR experiences, rendering dynamic PAF and force direction is essential. Although previous works have proposed the concept of dynamic PAF, the PAF is only in a limited space and without dynamic force direction. Therefore, we propose a controller, transPAF, to render omnidirectional impact feedback with dynamic PAF all round the controller for versatile VR scenarios. transPAF consists of a controller, a semicircular track, a linear track, and an impactor, which are all rotatable. The impactor can move to any position in a sphere, which means the whole 3D space all round the controller, and rotate in any direction. Therefore, dynamic PAF and force direction are achieved and independent to each other. We conducted a just-noticeable difference (JND) study to understand users’ distinguishability in position and direction, separately. A VR experience study was further performed to verify that feedback from transPAF with dynamic PAF and force direction enhances the VR experiences and demonstrate some applications for transPAF.

Rendering instant and intense impact feedback on users’ hands, limbs and head to enhance realism in virtual reality (VR) has been proposed in previous works, but impact on the body is still less discussed. With the body’s large surface area to utilize, numerous impact patterns can be rendered in versatile VR applications, e.g., being shot, blasted, punched or slashed on body in VR games. Herein we propose ImpactVest to render spatio-temporal multilevel impact force feedback on body. By independently controlling nine impactors in a 3 × 3 layout using elastic force, impact is generated at different levels, positions and time sequences for versatile spatial and temporal combinations. We conducted a just-noticeable difference (JND) study to understand users’ impact level distinguishability on the body. A time interval threshold study was then performed to ascertain what time interval thresholds between two impact stimuli should be used to distinguish from simultaneous impact, a continuous impact stroke and two discrete impact stimuli. Based on the results, we conducted a VR experience study to verify that impact feedback from ImpactVest enhances VR realism.

Interacting with not only virtual but also real objects, or even virtual objects augmented by real objects becomes a trend of virtual reality (VR) interactions and is common in augmented reality (AR). However, current haptic shape rendering devices generally focus on feedback of virtual objects, and require the users to put down or take off those devices to perceive real objects. Therefore, we propose FingerX to render haptic shapes and enable users to touch, grasp and interact with virtual and real objects simultaneously. An extender on the fingertip extends to a corresponding height to support between the fingertip and the real objects or the hand, to render virtual shapes. A ring rotates and withdraws the extender behind the fingertip when touching real objects. By independently controlling four extenders and rings on each finger with the exception of the pinky finger, FingerX renders feedback in three common scenarios, including touching virtual objects augmented by real environments (e.g., a desk), grasping virtual objects augmented by real objects (e.g., a bottle) and grasping virtual objects in the hand. We conducted a shape recognition study to evaluate the recognition rates for these three scenarios and obtained an average recognition rate of 76.59% with shape visual feedback. We then performed a VR study to observe how users interact with virtual and real objects simultaneously and verify that FingerX significantly enhances VR realism, compared to current vibrotactile methods.

Haptic feedback not only enhances immersion in virtual reality (VR) but also delivers experts’ haptic sensation tips in VR training, e.g., properly clamping a tenon and mortise joint or tightening a screw in the assembly of VR factory training, which could even improve the training performance. However, various and complicated manipulation is in different scenarios. Although haptic feedback of virtual objects’ shape, stiffness or resistive force in pressing or grasping is achieved by previous research, rotational resistive force when twisting or turning virtual objects is seldom discussed or explored, especially for a wearable device. Therefore, we propose a wearable device, ELAXO, to integrate continuous resistive force and continuous rotational resistive force with or without resilience in grasping and twisting, respectively. ELAXO is an exoskeleton with rings, mechanical brakes and elastic bands. The brakes achieve shape rendering and switch between with and without resilience modes for the resistive force. The detachable and rotatable rings and elastic bands render continuous resistive force in grasping and twisting. We conducted a just noticeable difference (JND) study to understand users’ distinguishability in the four conditions, resistive force and rotational resistive force with and without resilience, separately. A VR study was then performed to verify that the versatile resistive force feedback from ELAXO enhances the VR experiences.

Current haptic devices are usually designed to provide one type of force feedback; however, most VR scenarios require versatile force feedback, which may require the integration of different devices to provide various types of forces. In addition, besides the main haptic effects caused by the forces, multiple types of oscillation may also commonly accompany them, which are crucial for improving VR realism and immersion. Therefore, we simulate versatile force feedback by rendering the corresponding types of oscillation as the effects caused by those forces. We take inertia and impact forces as examples in this paper, and achieve versatility using the proposed device, OsciHead, on a head-mounted display (HMD), instead of integrating different devices. By controlling elastic bands' elasticity and stored power, OsciHead uses two rotatable oscillators on both sides of the HMD, in order to render various multilevel and multidimensional oscillation feedback in 2D translation and 2D rotation directions on a head. In an exploratory study, we explored different scenarios in which multiple types of oscillation could be simulated by OsciHead. We then observed oscillation level distinguishability in two just-noticeable difference (JND) studies, and evaluated the oscillation type recognition rates in a recognition study. Based on the results, we performed a VR study, which verified that the inertia and impact feedback simulated by OsciHead enhances realism and achieves versatility.

Many haptic feedback methods have been proposed to enhance realism in virtual reality (VR). However, friction on the feet in VR, which renders feedback as if walking on different terrains or ground textures or stepping on objects is still less explored. Herein, we propose a wearable device, FrictShoes a pair of foot accessories, to provide multilevel nonuniform friction feedback to feet. This is achieved by the independent functioning of six brakes on six wheels underneath each FrictShoe, which allows the friction levels of the wheels from each to be either matched or to vary. We conducted a magnitude estimation study to understand users' distinguishability of friction force magnitudes (or levels). Based on the results, we performed an exploratory study to realize how users adjust and map the multilevel nonuniform friction patterns to common VR terrains or ground textures. Finally, a VR experience study was conducted to evaluate the performance of the proposed multilevel nonuniform friction feedback to the feet in VR experiences.

Tactile feedback is widely used to enhance realism in virtual reality (VR). When touching virtual objects, stiffness and roughness are common and obvious factors perceived by the users. Furthermore, when touching a surface with complicated surface structure, differences from not only stiffness and roughness but also surface height are crucial. To integrate these factors, we propose a pin-based handheld device, HairTouch, to provide stiffness differences, roughness differences, surface height differences and their combinations. HairTouch consists of two pins for the two finger segments close to the index fingertip, respectively. By controlling brush hairs’ length and bending direction to change the hairs’ elasticity and hair tip direction, each pin renders various stiffness and roughness, respectively. By further independently controlling the hairs’ configuration and pins’ height, versatile stiffness, roughness and surface height differences are achieved. We conducted a perception study to realize users’ distinguishability of stiffness and roughness on each of the segments. Based on the results, we performed a VR experience study to verify that the tactile feedback from HairTouch enhances VR realism.

For haptic guidance, vibrotactile feedback is a commonly-used mechanism, but requires users to interpret its complicated patterns especially in 3D guidance, which is not intuitive and increases their mental effort. Furthermore, for haptic guidance in virtual reality (VR), not only guidance performance but also realism should be considered. Since vibrotactile feedback interferes with and reduces VR realism, it may not be proper for VR haptic guidance. Therefore, we propose a wearable device, GuideBand, to provide intuitive 3D multilevel force guidance upon the forearm, which reproduces an effect that the forearm is pulled and guided by a virtual guider or telepresent person in VR. GuideBand uses three motors to pull a wristband at different force levels in 3D space. Such feedback usually requires much larger and heavier robotic arms or exoskeletons. We conducted a just-noticeable difference study to understand users’ force level distinguishability. Based on the results, we performed a study to verify that compared with state-of-the-art vibrotactile guidance, GuideBand is more intuitive, needs a lower level of mental effort, and achieves similar guidance performance. We further conducted a VR experience study to observe how users combine and complement visual and force guidance, and prove that GuideBand enhances realism in VR guidance.

Force feedback from damped oscillation is a common effect in our daily lives, especially when shaking an elastic object, an object hanging or containing other stuff, or a container with liquid, e.g., casting with a fishing pole or wine-swirling. Such a force, affected by complex physical variations and collisions, is difficult to properly simulate using current force feedback methods. Therefore, we propose ElastOscillation on a virtual reality (VR) controller to provide 3D multilevel force feedback for damped oscillation to enhance VR experiences. ElastOscillation consists of a proxy, six elastic bands and DC motors. It leverages the motors to control the bands' elasticity to restrain the movement of the proxy, which is connected with the bands. Therefore, when users shake the ElastOscillation device, the proxy shakes or moves in corresponding ranges of movement. The users then perceive the force from oscillation at different levels. In addition, elastic force from the bands further reinforces the oscillation force feedback. We conducted a force perception study to understand users' distinguishability for perceiving oscillation forces in 1D and 2D movement, respectively. Based on the results, we performed a VR experience study to show that the force feedback provided by ElastOscillation enhances VR realism.

Force feedback is commonly used to enhance realism in virtual reality (VR). However, current works mainly focus on providing different force types or patterns, but do not investigate how a proper point of application of force (PAF), which means where the resultant force is applied to, affects users' experience. For example, users perceive resistive force without torque when pulling a virtual bow, but with torque when pulling a virtual slingshot. Therefore, we propose a set of handheld controllers, ElastiLinks, to provide force feedback between controllers with dynamic PAFs.A rotatable track on each controller provides a dynamic PAF, and two common types of force feedback, resistive force and impact, are produced by two links, respectively. We performed a force perception study to ascertain users' resistive and impact force level distinguishability between controllers. Based on the results, we conducted another perception study to understand users' distinguishability of PAF offset and rotation differences. Finally, we performed a VR experience study to prove that force feedback with dynamic PAFs enhances VR experience.

Resistive force (e.g., due to object elasticity) and impact (e.g., due to recoil) are common effects in our daily life. However, resistive force continuously changes due to users' movements while impact instantly occurs when an event triggers it. These feedback are still not realistically provided by current VR haptic methods. In this paper, a wearable device, ElasticVR, which consists of an elastic band, servo motors and mechanical brakes, is proposed to provide the continuously-changing resistive force and instantly-occurring impact upon the user's hand to enhance VR realism. By changing two physical properties, length and extension distance, of the elastic band, ElasticVR provides multilevel resistive force with no delay and impact with little delay, respectively, for realistic and versatile VR applications. A force perception study was performed to observe users' force distinguishability of the resistive force and impact, and the prototype was built based on its results. A VR experience study further proves that the resistive force and impact from ElasticVR both outperform those from current approaches in realism. Applications using ElasticVR are also demonstrated.

Impact is a common effect in both daily life and virtual reality (VR) experiences, e.g., being punched, hit or bumped. Impact force is instantly produced, which is distinct from other force feedback, e.g., push and pull. We propose ElastImpact to provide 2.5D instant impact on a head-mounted display (HMD) for realistic and versatile VR experiences. ElastImpact consists of three impact devices, also called impactors. Each impactor blocks an elastic band with a mechanical brake using a servo motor and extending it using a DC motor to store the impact power. When releasing the brake, it provides impact instantly. Two impactors are affixed on both sides of the head and connected with the HMD to provide the normal direction impact toward the face (i.e., 0.5D in z-axis). The other impactor is connected with a proxy collider in a barrel in front of the HMD and rotated by a DC motor in the tangential plane of the face to provide 2D impact (i.e., xy-plane). By performing a just-noticeable difference (JND) study, we realize users' impact force perception distinguishability on the heads in the normal direction and tangential plane, separately. Based on the results, we combine normal and tangential impact as 2.5D impact, and performed a VR experience study to verify that the proposed 2.5D impact significantly enhances realism.

One-handed input for smartwatches is crucial when users’ hands are occupied. A rotational motion is leveraged, which can be detected by built-in motion sensors in most smartwatches, as input to provide item selection. Users control a cursor by rotating the smartwatch into different orientations using the proposed item selection gestures. To understand human wrist dexterity and the ability to perform rotational motion input along each degree of freedom on smartwatches, a human-factor study was performed. Except dexterity, users’ consensus and agreement of user-defined rotational motion input gestures for selection and commitment on smartwatches in an exploratory user study were also observed. Based on both the studies, gestures roll and tilt are proposed as selection gestures, and shake and nod are used as commitment gestures. The item selection performance using the proposed gestures in 2D layouts in a user study was further evaluated. The study results showed that tilt-nod and tilt-shake are suitable for item numbers (4 × 3) and (5 × 5), respectively. Furthermore, due to high error rate for tilt-shake in (2 × 2) caused by false positive in shake detection, roll-shake with better social acceptance can be used as an alternative for smartwatches without a built-in front camera in (2 × 2).

We propose light-weight wearable devices, iKneeBraces, to prevent knee osteoarthritis (OA) using knee adduction moment (KAM) evaluation. iKneeBrace consists of two inertial measurement units (IMUs) to measure shin and thigh angles. KAM is estimated by ground force reaction (GRF), knee position and center of pressure position. Instead of heavy and bulky 3DoF force plates conventionally used, we propose to build a 2D input regression model using shin and thigh angles from iKneeBrace as input to infer GRF direction and further estimate KAM. We perform an experiment to evaluate the method. The results show that iKneeBrace can infer KAM similar to the ground truth in the first peak, the most important part to prevent knee OA. Furthermore, the proposed method can infer KAM in all parts if better IMUs used in iKneeBrace in the future. The proposed method not only makes KAM evaluation portable but also requires only light-weight devices.

For makers and developers, circuit prototyping is an integral part of building electronic projects. Currently, it is common to build circuits based on breadboard schematics that are available on various maker and DIY websites. Some breadboard schematics are used as is without modification, and some are modified and extended to fit specific needs. In such cases, diagrams and schematics merely serve as blueprints and visual instructions, but users still must physically wire the breadboard connections, which can be time-consuming and error-prone. We present CircuitStack, a system that combines the flexibility of breadboarding with the correctness of printed circuits, for enabling rapid and extensible circuit construction. This hybrid system enables circuit reconfigurability, component reusability, and high efficiency at the early stage of prototyping development.

Glint features have important roles in gaze tracking systems. But when the operation range of a gaze tracking system is enlarged, the performance of glint-feature-based (GFB) approaches will be degraded mainly due to the curvature variation problem at around the edge of the cornea. Although the pupil contour feature may provide complementary information to help estimating the eye gaze, existing methods do not properly handle the cornea refraction problem, leading to inaccurate results. This paper describes a contour-feature-based (CFB) 3-D gaze tracking method that is compatible to cornea refraction. Experiments show the effectiveness of the proposed approach for gaze tracking.