Manual precise manipulation of objects is a critical skill in daily life, and Augmented Reality (AR) is increasingly used to support such tasks. In this study, we propose a system utilizing pseudo-haptic feedback to support precise manipulation for six degrees of freedom (6DOF). Two types of AR instruction interfaces were developed: Visual Deviation Instruction Interface (VDI) and Pseudo-Haptic Instruction Interface (PHI). A user study with 18 participants compared the two instruction interfaces in terms of performance and user experience. The objective measures of performance (task completion time, deviation), and the subjective measures (system usability scale, NASA Task Load Index) were collected. Results show that both instruction interfaces effectively support manual precise manipulation, achieving position deviations under 2 mm and orientation deviations under 1°. PHI outperformed VDI in speed, mental effort, physical demand, performance, perceived workload, custom user experience elements and reduction deviations of manual precise manipulation. Finally, we discuss research limitations and future directions.

Current Virtual Reality (VR) sketching systems lack natural bimanual interactions for drawing curves on 3D models. This article introduces HoldTabSketch, a VR sketching system that uses a tracked graphics tablet as a touch-sensitive physical proxy to enable realistic bimanual interactions for 3D curve drawing. The system features a Reoriented Bounding Boxes (ROBB) haptic redirection technique that automatically aligns virtual pen and hand movements with various virtual objects while maintaining tactile feedback. The system supports two additional bimanual interaction modes: 1) bare-hand object manipulation with pen-based sketching using visual feedback, and 2) tablet-assisted tangible interactions combining grasp support and surface drawing. A 12-user comparative study showed the tablet-based physical proxy significantly improved the realism of 3D object manipulation and surface drawing, yielding higher user preference and usability compared to vision-only or basic tangible approaches. The system demonstrates how hybrid physical-digital proxies can enhance 3D creative workflows in VR.

Virtual content in augmented reality (AR) applications can be tailored to a designer's specifications. However, real-world environments are challenging to control precisely or replicate fully. Consequently, prototyping AR applications for specific environments is often difficult. One potential solution is employing mixed reality (MR) to simulate an AR system, enabling controlled experiments. Nevertheless, the effectiveness of using MR to simulate AR office work remains underexplored. In this paper, we report the results of a user study (N = 40) that investigated the impact of an MR simulation of an AR office on participants' task performance and cognitive workload (CWL). Participants completed several office tasks in both an AR scene featuring a virtual monitor and an MR-simulated AR scene. During these tasks, CWL was measured using electroencephalography (EEG) and a subjective questionnaire. The results show that the performance of the pass-through window is a major constraint on the effectiveness of the MR simulation office. Finally, we discuss the study's limitations and directions for future research.

Augmented reality (AR) can bring a new interactive experience to the collaboration between users. When users are in the same place, there are two modes of joint operation for the same object: parallel-work (PW) and cross-work (CW). PW means two users perform their tasks, while CW means assisting each other. To investigate the difference that collaboration using PW and CW in an AR environment brings to users, we developed a two-person local collaboration system, LoCol. We designed and conducted user experiments by selecting the tasks of adjusting the virtual model of the assembly and adding missing boundaries in the model. The results showed that CW led to a higher sense of social coexistence while reducing workload. In terms of task completion time and accuracy, CW and PW each had advantages. We found that users generally want to reduce unnecessary repetitive operations and frequent movement by working with others. This is likely an important criterion for determining who is better suited for a particular job in either approach.

The shortcomings of established input methods for Augmented Reality (AR) head-mounted displays (HMDs) motivate us to investigate the use of AR HMDs with smartphones and smartwatches to improve AR interaction in physical tasks. However, it is unclear whether the cross-device interaction interfaces are efficient for AR systems on physical tasks because the physical tasks can break the interaction flow. In this work, we conducted a user study to explore this. The user study consists of three subtasks, respectively requiring one of the three representative user interface (UI) controls (buttons, sliders, or text input). We implemented them with mid-air gestures, a smartphone, and a smartwatch. We compared the three interfaces and found that the smartphone and the smartwatch interaction interfaces have greater usability, provide a better user experience, and have lower workloads than the mid-air gesture interface. However, there is no significant difference in terms of efficiency for simple interactions. We discuss the limitations of this research and directions for future work.

Interaction with virtual objects is one of the essential features of Augmented Reality (AR) systems. One of its main issues is how to provide precise manipulation of distant virtual objects in AR. In this work, we explore bare-hand manipulation of distant objects in AR with DOF (degree-of-freedom) separation, motion scaling, and near-field metaphors. We developed two manipulation techniques: the distant widget-based metaphor (DWBM), and the near-field widget-based metaphor (NFWBM). We conducted a user study with 20 participants to compare the two techniques in terms of performance and user experience. We found that NFWBM has faster speed, lower mental effort, better ease-of-use, and a friendlier user experience. However, there is no significant difference in terms of precision; both techniques could manipulate distant objects precisely, with an average position error of less than 0.7 cm and an average orientational error of less than 1°. We also discussed the limitations of this research and directions for future work.

The projection-augmented reality assembly assistance system, which supports natural multi-modal human–computer interaction, has the potential to improve interaction efficiency. However, the effectiveness of the interaction method is compromised by the interference caused by the dynamically changing industrial environment. To address this issue, this study proposed a multi-model interaction model based on fuzzy theory and developed an environment perception decision-making interaction system. This system continuously monitors real-time environment data, evaluates interaction performance, and visually recommends the optimal method to assist users in completing their tasks. A comparative experiment was conducted between the proposed system and a non-environment perception interaction system. The collected data, including interaction success rate, task time, system usability, and user experience, were analyzed. The results demonstrated that the proposed system effectively enhances the adaptability and comfort of computer systems in complex environments and improves communication efficiency between humans and machines.

Augmented Reality (AR) based manual assembly nowadays enables to guide the process of physical tasks, providing intuitive instructions and detailed information in real-time. However, very limited studies have explored AR manual adjustment tasks with precision requirements. In this paper, we develop an AR-assisted guidance system for manual adjustments with relatively high-precision requirements. We first assessed the accuracy of the special-set OptiTrack system to determine the threshold of precision requirements for our user study. We further evaluated the performance of Number-based and Bar-based precision feedback by comparing orienting assembly errors and task completion time, as well as the usability in the user study. We found that the assembly errors of orientation in the Number-based and Bar-based interfaces were significantly lower than the baseline condition, while there was no significant difference between the Number-based and Bar-based interfaces. Furthermore, the Number-based showed faster task completion time, lower workload, and higher usability than the Bar-based condition.

During component assembly, some operations must be completed by two or more workers due to the size or assembly mode. For instance, in manual riveting, two workers are positioned on either side of a steel plate, which blocks the view. Traditional collaborative approaches limit assembly efficiency and is difficult to ensure accurate and rapid interaction between workers. In this study, we developed an AR-Assisted Cooperative Assembly System (ARCoA) to address the issue. ARCoA allows users to view their partner’s key information that is occluded, including tools, gestures, orientations, and shared markers. Besides, we presented a user experiment that compared this method with the traditional approach. The results indicated that the new system could significantly improve assembly efficiency, system availability, and sense of social presence. Moreover, most users we surveyed preferred ARCoA. In the future, we will incorporate more functions and improve the accuracy of the system to solve complex multi-person collaborative problems.

Consistent visual and haptic feedback is an important way to improve the user experience when interacting with virtual objects. However, the perception provided in Augmented Reality (AR) mainly comes from visual cues and amorphous tactile feedback. This work explores how to simulate positional vibrotactile feedback (PVF) with multiple vibration motors when colliding with virtual objects in AR. By attaching spatially distributed vibration motors on a physical haptic proxy, users can obtain an augmented collision experience with positional vibration sensations from the contact point with virtual objects. We first developed a prototype system and conducted a user study to optimize the design parameters. Then we investigated the effect of PVF on user performance and experience in a virtual and real object alignment task in the AR environment. We found that this approach could significantly reduce the alignment offset between virtual and physical objects with tolerable task completion time increments. With the PVF cue, participants obtained a more comprehensive perception of the offset direction, more useful information, and a more authentic AR experience.

With the application of artificial intelligence technology in the medical field, the use of deep learning methods to predict lymph node (LN) metastasis from Computed Tomography (CT) images has become one of the important studies in adjuvant cancer treatment. Although many studies have made some progress, the large difference in LNs size has always limited the performance of traditional convolutional neural network methods. In this paper, we propose a multi-scale attention fully convolutional network for LN metastasis prediction in CT images. We aim to make the network compatible with LNs of different sizes to improve the overall performance of the network. First, the network extracts image features from multiscale input data. Then, we use an attention-based fusion module to study the relationship between features at different scales and adaptively fuse image features. Furthermore, a feature consistency loss is proposed by us to enhance the similarity between image features at different scales. The experimental results show that our proposed network achieves the best performance with the accuracy 92.37%, the sensitivity 90.21% and the specificity 95.71%, which outperforms several state-of the-art methods.

In AR assembly, only experienced workers can complete the physical tasks with high operation precision requirements. The reason is that they use their brains to transform the information reflecting micro-operation intention into a series of manual operation rules. Because novices are not familiar with these rules, they often encounter problems of low precision and low efficiency. In order to describe these rules with AR instruction, this paper does some research work. Firstly, traditional AR and microAR are defined, and their differences and relations in guiding micro-operation are explained. On this basis, a microAR oriented to human microoperation rules is proposed. Then, according to the traditional AR and microAR, three instructions are designed to guide novice micro-operation assembly. An example is designed to test the assembly time and operation experience of three interfaces under a physical task. The results show that compared with traditional AR instruction, microAR instruction can better guide novice micro-operation.

We propose Fluid3DGuides, a drawing guide technique to help users draw structured sketches more accurately in VR. The prototype system continuously infers visual guide lines for the user based on the user’s instant stroke drawing intention and its potential constraint relationship with the existing strokes. We evaluated our prototype through a pilot user study with six participants by comparing the proposed guide technique against the non-guide drawing condition. Participants gave positive comments on ease of use and drawing accuracy. They found that the technique could reduce the time and effort required to find the corrected drawing perspective and obtain more accurate 3D structured sketches.

The research of augmented reality physical tasks for human-computer collaboration is still an attractive dynamic research field. In recent years, AR Instruction supporting collaborative assembly has been developed, which can transfer graphic annotation between man and machine. However, there is little research on high precision manual assembly. This paper proposes a high-precision cognitive AR representation (M-AR) for manual operation guidance, which aims to transform a high-precision assembly relationship into a series of visual information to meet the user’s cognition of high-precision task intention. Using this representation, we conducted a within-subject case study to compare it with original AR representation (original AR) and low-precision cognitive AR representation. The results showed that there were significant differences in execution time and operation errors. The user’s subjective feedback also showed that M-AR supporting cognitive feedback can significantly improve a user’s cognitive experience in a physical task.

When interacting with virtual objects, we could discover some problems around us. Digital information has no physical properties, and sense organ(vision, tactile) does not match very well. These have a negative influence on the user’s interactive experience. In this paper, we propose to provide virtual objects with no physical properties with physical affordances to support tangible feedback via mechanical movement. In addition, we developed an interaction prototype system. The system could change the physical supports that are absorbed onto an electromagnet to adapt to the shape of different virtual objects and efficiently provide natural and consistent interactions when putting physical objects onto virtual objects in Augmented Reality (AR) scenarios.

Abstract This paper investigates the effect of using augmented reality (AR) annotations and two different gaze visualizations, head pointer (HP) and eye gaze (EG), in an AR system for remote collaboration on physical tasks. First, we developed a spatial AR remote collaboration platform that supports sharing the remote expert’s HP or EG cues. Then the prototype system was evaluated with a user study comparing three conditions for sharing non-verbal cues: (1) a cursor pointer (CP), (2) HP and (3) EG with respect to task performance, workload assessment and user experience. We found that there was a clear difference between these three conditions in the performance time but no significant difference between the HP and EG conditions. When considering the perceived collaboration quality, the HP/EG interface was statistically significantly higher than the CP interface, but there was no significant difference for workload assessment between these three conditions. We used low-cost head tracking for the HP cue and found that this served as an effective referential pointer. This implies that in some circumstances, HP could be a good proxy for EG in remote collaboration. Head pointing is more accessible and cheaper to use than more expensive eye-tracking hardware and paves the way for multi-modal interaction based on HP and gesture in AR remote collaboration.

Haptic guidance is an essential research topic of haptic interaction technology. In some conditions, haptic guidance devices could be better at assisting navigation. We developed Haptic Compass, a prototype with active vibrotactile feedback by attaching vibration motors to a cylinder-shaped physical object, to provide directional cues for haptic guidance. To validate the prototype, we conducted two user studies. The first study investigated if users could identify the vibrotactile feedback from the prototype. The results showed that participants could effectively recognize and judge the vibration direction. In the second study, we evaluated the task completion time and the movement error among the Visual-only, Haptic-Only and Haptic-Visual feedback conditions in a typical path-guiding task. We found that, together with visual feedback, haptic feedback could enhance task performance and user experience for direct path guidance. Performance of the two studies demonstrated that our prototype has the potential to be an alternative for applications where non-wearable haptic feedback is desired.

This paper investigates how two different unique gaze visualizations (the head pointer(HP), eye gaze(EG)) affect table-size physical tasks in Mixed Reality (MR) remote collaboration. We developed a remote collaborative MR Platform which supports sharing of the remote expert's HP and EG. The prototype was evaluated with a user study comparing two conditions: sharing HP and EG with respect to their effectiveness in the performance and quality of cooperation. There was a statistically significant difference between two conditions on the performance time, and HP is a good proxy for EG in remote collaboration.

Haptic feedback has shown its great value in HCI research and applications for enhancing user experiences. Simulating people's tactile sensations of virtual objects is currently a primary research target. Different from wearing motors on fingers or hands, we attached vibration motors on a physical object to simulate the augmented sense of collision with virtual objects to bare hands. We developed a novel sensor-based proof-of-concept prototype that distributes multiple vibration motors around a physical object and provides vibrational sensations from the collision direction through combinations of motors. Users can obtain augmented haptic feedback when manipulating the augmented physical object to interact with virtual objects in an AR environment. We first studied the influence of the number and input voltage of motors for a correct judgment of different directions to identify the design parameters of the prototype. Then we investigated the effect of introducing the prototype in a typical manipulation task in AR. We found the prototype was efficient for enhancing human performance and collision experience with virtual objects together with visual feedback.

We introduce new methods to replay intensive block I/O workloads more accurately. These methods can be used to reproduce realistic workloads for benchmarking, performance validation, and tuning of a high-performance block storage device/system. In this article, we study several sources in the stock operating system that introduce uncertainty in the workload replay. Based on the remedies of these findings, we design and develop a new replay tool called hfplayer that replays intensive block I/O workloads in a similar unscaled environment with more accuracy. To replay a given workload trace in a scaled environment with faster storage or host server, the dependency between I/O requests becomes crucial since the timing and ordering of I/O requests is expected to change according to these dependencies. Therefore, we propose a heuristic way of speculating I/O dependencies in a block I/O trace. Using the generated dependency graph, hfplayer tries to propagate I/O related performance gains appropriately along the I/O dependency chains and mimics the original application behavior when it executes in a scaled environment with slower or faster storage system and servers. We evaluate hfplayer with a wide range of workloads using several accuracy metrics and find that it produces better accuracy when compared to other replay approaches.

The demand for parallel I/O performance continues to grow. However, modelling and generating parallel I/O work-loads are challenging for several reasons including the large number of processes, I/O request dependencies and workload scalability. In this paper, we propose the PIONEER, a complete solution to Parallel I/O workload characterization and gEnERation. The core of PIONEER is a proposed generic workload path, which is essentially an abstract and dense representation of the parallel I/O patterns for all processes in a High Performance Computing (HPC) application. The generic workload path can be built via exploring the inter-processes correlations, I/O dependencies as well as file open session properties. We demonstrate the effectiveness of PIONEER by faithfully generating synthetic workloads for two popular HPC benchmarks and one real HPC application.

Shingled write disk (SWD) is a magnetic hard disk drive that adopts the shingled magnetic recording (SMR) technology to overcome the areal density limit faced in conventional hard disk drives (HDDs). The SMR design enables SWDs to achieve two to three times higher areal density than the HDDs can reach, but it also makes SWDs unable to support random writes/in-place updates with no performance penalty. In particular, a SWD needs to concern about the random write/update interference, which indicates writing to one track overwrites the data previously stored on the subsequent tracks. Some research has been proposed to serve random write/update out-of-place to alleviate the performance degradation at the cost of bringing in the concept of garbage collection. However, none of these studies investigate SWDs based on the garbage collection performance. In this paper, we propose a SWD design called Hot data identification-based Shingled Write Disk (H-SWD). The H-SWD adopts a window-based hot data identification to effectively manage data in the hot bands and the cold bands such that it can significantly reduce the garbage collection overhead while preventing the random write/update interference. The experimental results with various realistic workloads demonstrates that H-SWD outperforms the Indirection System. Specifically, incorporating a simple hot data identification empowers the H-SWD design to remarkably improve garbage collection performance.

In this paper we model and analyze a cross-layer integrated mobility and service management scheme called DMAPwSR in Mobile IPv6 environments with the goal to minimize the overall mobility and service management cost for serving mobile users with diverse mobility and service characteristics. The basic idea of DMAPwSR is that each mobile node (MN) can utilize its application-layer knowledge to choose smart routers to be its dynamic mobility anchor points (DMAPs) to balance the cost associated with mobility services vs. packet delivery services. These smart routers are just access routers for MIPv6 systems except that they are capable of processing binding messages from the MN and storing the current location of the MN in the routing table for forwarding service packets destined to the MN. The MN's DMAP changes dynamically as the MN roams across the MIPv6 network. Furthermore the DMAP service area also changes dynamically reflecting the MN's mobility and service behaviors dynamically. Unlike previous mobility management protocols such as HMIPv6 that focus only on mobility management, DMAPwSR considers integrated mobility and service management. We develop an analytical model based on stochastic Petri nets to analyze DMAPwSR and compare its performance against MIPv6 and HMIPv6. We validate analytical solutions obtained through extensive simulation including sensitivity analysis of simulation results with respect to the network coverage model, the MN's residence time distribution and the DMAP service area definition.

In this paper, we investigate a proxy-based integrated cache consistency and mobility management scheme in mobile IP systems. Our scheme is based on a stateful strategy by which cache invalidation messages are asynchronously sent by the server to a mobile host (MH) whenever data objects cached at the MH have been updated. We use a per-user proxy to buffer invalidation messages to allow the MH to disconnect arbitrarily and to reduce the number of uplink requests when the MH is reconnected. Moreover, the MH's proxy serves as a gateway foreign agent (GFA) as in the MIP Regional Registration protocol to keep track of the address of the MH in a region. The proxy migrates with the MH when the MH crosses a regional area. We identify the optimal regional area size under which the overall network traffic cost, due to cache consistency management, mobility management, and query requests/replies, is minimized. We demonstrate that the integrated cache consistency and mobility management scheme outperforms both no-proxy and/or no-cache schemes in Mobile IPv6 environments.

We propose and analyse a proxy-based regional registration scheme for integrated mobility and service management with the goal to minimize the network signaling and packet delivery cost in Mobile Internet Protocol (MIP) systems. Under the proposed proxy-based regional registration scheme, a client-side proxy is created on a per-user basis to serve as a gateway between a mobile node (MN) and all services engaged by the MN. From the perspective of these services, the proxy behaves as if it were the MN. From the perspective of the MN, the proxy behaves as if it were the services. Leveraging MIP with route optimization, the proxy runs on a foreign agent node and cooperates with the home agent and foreign agents of the MN in the MIP network to maintain the location information of the MN in order to facilitate data delivery by services engaged by the MN. Further the proxy can optimally determine when to move with the MN so as to minimize the network cost associated with the user's mobility and service management. We investigate the notion of ‘service areas’ for the proxy to perform ‘service handoffs’ in our scheme. We show that, when given a set of parameters characterizing the operational and workload conditions of an MN, there exists an optimal service area size for the MN such that the network communication cost is minimized for serving mobility and service management operations of the MN. We demonstrate via Petri net modeling and analysis that our proposed scheme outperforms both basic MIP and MIP regional registration that do not consider integrated mobility and service management.

We investigate an integrated mobility and service management scheme based on MIPv6 with the goal to minimize the overall network signaling cost in MIPv6 systems for serving mobility and service management related operations. Our design extends hierarchical mobile IPv6 (HMIPv6) with the notion of dynamic mobility anchor points (DMAPs) for each mobile node (MN) instead of static ones for all MNs. These DMAPs are access routers (ARs) chosen by individual MNs to act as a regional router to reduce the signaling overhead for intra-regional movements. The DMAP domain size, i.e., the number of subnets covered by a DMAP, is based on the MN's mobility and service characteristics. Under our DMAP protocol, a MN optimally determines when and where to launch a DMAP to minimize the network cost in serving the user's mobility and service management operations. We demonstrate that our DMAP protocol for integrated mobility and service management yields significantly improved performance over basic MIPv6 and HMIPv6

Mobile commerce (m-commerce) refers to the conduct of business using wireless devices and communications. Driven by the success of e-commerce and impressive progress in wireless technologies, m-commerce is rapidly taking place in the business forefront. However, most of the concepts developed for e-commerce may not be easily applicable to wireless environments. This is due to the peculiarities of these environments such as limited bandwidth, unbalanced client-server communication, and limited power supply. Web services are undeniably one of the most significant e-commerce concepts worth of being adapted to the wireless world. Mobile services, also called m-services, promise several benefits compared with their wired counterparts. They provide larger customer base and cater for "anytime and anywhere" access to services. In this paper, we propose an infrastructure for organizing and efficiently accessing m-services in broadcast environments. We define a multichannel model to carry information about m-services available within a given geographic area. The UDDI channel includes registry information about m-services. The m-service channel contains the description and executable code of each m-service. The data channel contains the actual data needed while executing the m-service. We also introduce three techniques to enable efficient access to wireless channels. These techniques extend well-known mobile databases' access methods to m-services: B+ tree, signature indexing, and hashing. We finally present an analytical model and conduct an extensive experimental study to evaluate and compare the proposed techniques.