Requirements workshop for software stakeholders is the major approach for collecting software requirements. Given the success of AI (Artificial Intelligence), especially NLP (Natural Language Processing) and the recent LLMs (Large Language Models), it is becoming increasingly popular for requirements engineers to use cloud-based AI services to automate the process of requirements documentation based on manually or AI generated meeting minutes either during or after the requirements workshop. However, using cloud-based AI services to process audio recordings of the requirements workshop may introduce many issues such as data privacy, high demand for computational resources and bandwidth, and large latency which make collaboration not efficient. To address these issues, we propose Edge4RE, a novel privacy-aware edge-cloud collaborative framework to support automated software requirements documentation from automated audio recordings during the requirements workshop. Specifically, with our solution, automated audio recordings are first transcribed and segmented into sentences on the local edge device. The edge device applies a lightweight Transformer based classifier fine-tuned using Low Rank Adaptation (LoRA) to identify requirement-related sentences. Afterwards, only requirement-related sentences are sent to the cloud-based LLM to generate the structured requirements documentation. Experimental results demonstrate that our framework can achieve high-quality requirement documentation based on automated audio recordings while reducing latency and bandwidth consumption with privacy protection.

Accessibility is a crucial concept in Virtual Reality (VR), pivotal for meeting the needs of users, including those with disabilities.In recent years, there has been an increasing focus of VR products on enhancing the accessibility of a diverse range of digital content.Despite this growing attention from the VR community, there is a serious lack of empirical research on how VR practitioners consider VR accessibility.This includes their understanding of and insights into VR accessibility challenges and practices in the VR software development life cycle.In this paper, we aim to address these gaps using a mixed-methods approach.Specifically, we conducted interviews with 21 VR practitioners (incl.3D modelers, developers, technical directors, and product managers) and surveyed 202 VR practitioner respondents from VR-related industries.Our findings outline the insights and challenges they face concerning VR accessibility practices in the software development life cycle.Furthermore, our findings shed light on the challenges faced by practitioners concerning VR accessibility and the reasons why it often goes unconsidered.As far as we know, this is the first comprehensive report about the understanding of accessibility in the VR software development life cycle from practitioners' perspectives.We hope this paper will help VR practitioners better understand the practices, challenges, and potential solutions related to VR accessibility.

Telerobotics is a key foundation in autonomous Industrial Cyber-Physical Systems (ICPS), enabling remote operations across various domains. However, conventional cloud-based telerobotics suffers from latency, reliability, scalability, and resilience issues, hindering real-time performance in critical applications. Cloud-Fog Telerobotics (CFTel) builds on the Cloud-Fog Automation (CFA) paradigm to address these limitations by leveraging a distributed Cloud-Edge-Robotics computing architecture, enabling deterministic connectivity, deterministic connected intelligence, and deterministic networked computing. This paper synthesizes recent advancements in CFTel, aiming to highlight its role in facilitating scalable, low-latency, autonomous, and AI-driven telerobotics. We analyze architectural frameworks and technologies that enable them, including 5G Ultra-Reliable Low-Latency Communication, Edge Intelligence, Embodied AI, and Digital Twins. The study demonstrates that CFTel has the potential to enhance real-time control, scalability, and autonomy while supporting service-oriented solutions. We also discuss practical challenges, including latency constraints, cybersecurity risks, interoperability issues, and standardization efforts. This work serves as a foundational reference for researchers, stakeholders, and industry practitioners in future telerobotics research.

Industrial Cyber-Physical Systems (ICPS) technologies are foundational in driving maritime autonomy, particularly for Unmanned Surface Vehicles (USVs). However, onboard computational constraints and communication latency significantly restrict real-time data processing, analysis, and predictive modeling, hence limiting the scalability and responsiveness of maritime ICPS. To overcome these challenges, we propose a distributed Cloud-Edge-IoT architecture tailored for maritime ICPS by leveraging design principles from the recently proposed Cloud-Fog Automation paradigm. Our proposed architecture comprises three hierarchical layers: a Cloud Layer for centralized and decentralized data aggregation, advanced analytics, and future model refinement; an Edge Layer that executes localized AI-driven processing and decision-making; and an IoT Layer responsible for low-latency sensor data acquisition. Our experimental results demonstrated improvements in computational efficiency, responsiveness, and scalability. When compared with our conventional approaches, we achieved a classification accuracy of 86%, with an improved latency performance. By adopting Cloud-Fog Automation, we address the low-latency processing constraints and scalability challenges in maritime ICPS applications. Our work offers a practical, modular, and scalable framework to advance robust autonomy and AI-driven decision-making and autonomy for intelligent USVs in future maritime ICPS.

Context: Personas are commonly employed in software projects to better understand end-users needs. Despite their frequent usage, there is a limited understanding of their practical application and effectiveness. Objective: This paper aims to investigate the current practices, methods, and challenges associated with using personas in software development. Methods: A two-step investigation was conducted, comprising interviews with 26 software developers, UI/UX designers, business analysts, and product managers, along with a survey of 203 practitioners. Results: The findings reveal variations in the frequency and effectiveness of personas across different software projects and IT companies. Additionally, the study highlights the challenges practitioners face when using personas and the reasons for not using them. Notably, the research shows that some human aspects (e.g., the needs of users with disabilities), often assumed to be a key feature of personas, are frequently not considered for various reasons in requirements engineering. Conclusions: The study provides actionable insights for practitioners to overcome challenges in using personas during the requirements engineering stages. Furthermore, it identifies areas for future research to enhance the effectiveness of personas in software development.

Volumetric telepresence aims to create a shared space, allowing people in local and remote settings to collaborate seamlessly. Prior telepresence examples typically have asymmetrical designs, with volumetric capture in one location and objects in one format. In this paper, we present a volumetric telepresence mixed reality system that supports real-time, symmetrical, multi-user, partially distributed interactions, using objects in multiple formats, across multiple locations. We align two volumetric environments around a common spatial feature to create a shared workspace for remote and co-located people using objects in three formats: physical, virtual, and volumetric. We conducted a study with 18 participants over 6 sessions, evaluating how telepresence workspaces support spatial coordination and hybrid communication for co-located and remote users undertaking collaborative tasks. Our findings demonstrate the successful integration of remote spaces, effective use of proxemics and deixis to support negotiation, and strategies to manage interactivity in hybrid workspaces.

This paper presents a novel method for evaluating wayfinding within a public building to provide meaningful insights for stakeholders. Our approach features unique methods for both data collection and evaluation, with a holistic digital capture of the entire virtual environment experienced by participants, maintained in an interactive format for in-depth analysis. We also captured and output data in point cloud formats, raw data text files, and task-specific metrics, which support interactive replays of participants’ experiences. We developed algorithms to extract meaningful insights from the raw data based on assumptions about wayfinding characteristics. The contribution is a flexible framework that can be easily adapted for future projects with adjustable variables to suit specific applications.

Abstract Squamous cell carcinoma is the most common type of cancer that occurs in squamous cells of epithelial tissue. Histopathological evaluation of tissue samples is the gold standard approach used for carcinoma diagnosis. SCC detection based on various histopathological features often employs traditional machine learning approaches or pixel-based deep CNN models. This study aims to detect keratin pearl, the most prominent SCC feature, by implementing RetinaNet one-stage object detector. Further, we enhance the model performance by incorporating an attention module. The proposed method is more efficient in detection of small keratin pearls. This is the first work detecting keratin pearl resorting to the object detection technique to the extent of our knowledge. We conducted a comprehensive assessment of the model both quantitatively and qualitatively. The experimental results demonstrate that the proposed approach enhanced the mAP by about 4% compared to default RetinaNet model.

Mixed-reality telepresence allows local and remote users feel as if they are present together in the same space. In this paper we report on a mixed-reality volumetric telepresence system that is adaptable, multi-user and cross-modal, i.e. combining augmented and virtual reality technologies with face-to-face interactions. The system extends state-of-art by creating full-body and environmental volumetric renderings in real-time over local enterprise networks. We report findings of an evaluation in a training scenario which was adapted for remote delivery and led by an industry professional. Analysis of interviews and observed behaviours identify varying attitudes towards virtually mediated full-body experiences and highlight the impact of volumetric mixed-reality telepresence to facilitate personal experiences of co-presence and to ground communication with interlocutors.

People who use software applications are different, including with significant cognitive differences such as neurodiversity. Capturing requirements for software that addresses these cognitive differences is hard for software engineers, especially when they do not have the same cognitive challenges. We wanted to explore the use of virtual reality (VR) in assisting software engineers to better understand the perspectives of the end user for the purpose of human-centric requirements elicitation, with a focus on users with attention-deficit/hyperactivity disorder (ADHD). We developed an immersive VR prototype using a virtual gym environment and fitness app as a concrete motivating example scenario. We carried out an evaluation of requirements identification for ADHD fitness app users by instructing participants to complete activities whilst under visual and auditory distractions similar to various documented symptoms of ADHD. Results indicated an increase in understanding the perspectives of someone with ADHD and an awareness of potential challenges with software not intentionally designed for ADHD users. Improved requirements for our target fitness app resulted that better take into account these diverse user needs.

The increased availability of immersive headsets in the games industry has promoted wide adoption of immersive technology amongst consumers. The benefits of spatial freedom and agency for body movements in virtual reality pave the way for everyday fitness and physical training applications. We conducted a systematic review through ACM Digital Library, IEEEXplore, and Scopus, to investigate how immersive technologies have been applied for physical training in fitness and sports. Our review included all peer-reviewed papers, published and written in English, discussing all forms of hardware for VR, AR, and MR technologies, targeted towards healthy population. We excluded applications for clinical populations and treatment of specific diseases, and all non-peer reviewed articles like position papers, workshops, tutorials, and magazine columns. From the shortlisted 144 papers, we summarize the development trends of immersive technologies and highlight the challenges of designing immersive technologies for everyday fitness. We also provide recommendations for future work and highlight the need to support better collaboration with industry partners, trainer-trainee experiences, and social dynamics of sports for designing better experiences.

Requirements engineering (RE) plays a crucial role in developing software systems by bridging the gap between stakeholders' needs and system specifications. However, effective communication and elicitation of stakeholder requirements can be challenging, as traditional RE methods often overlook emotional cues. This paper introduces a multi-modal emotion recognition platform (MEmoRE) to enhance the requirements engineering process by capturing and analyzing the emotional cues of stakeholders in real-time. MEmoRE leverages state-of-the-art emotion recognition techniques, integrating facial expression, vocal intonation, and textual sentiment analysis to comprehensively understand stakeholder emotions. This multimodal approach ensures the accurate and timely detection of emotional cues, enabling requirements engineers to tailor their elicitation strategies and improve overall communication with stakeholders. We further intend to employ our platform for later RE stages, such as requirements reviews and usability testing. By integrating multi-modal emotion recognition into requirements engineering, we aim to pave the way for more empathetic, effective, and successful software development processes. We performed a preliminary evaluation of our platform. This paper reports on the platform design, preliminary evaluation, and future development plan as an ongoing project.

Virtual Reality (VR) installations using head-mounted displays (HMD) are becoming increasingly popular in public spaces. However, VR's immersive nature engages only the HMD wearer and excludes everyone else in the public space, and there is little design knowledge of how to engage those not wearing an HMD. To address this, we draw from our experiences of having designed seven public VR installations to present a design space around the dimensions of “agency” and “interest” with four user engagement frames to articulate twelve different user roles. To guide designers to support all roles and to transition users between those roles, we complement the design space with a set of design tactics for public VR installations. We hope that these combined contributions will help designers engage more people with VR installations so that ultimately more people benefit from what VR has to offer.

Virtual Reality (VR) challenges us with the requirements of ultra-low latency and ultra-high bandwidth. Existing methods that rely on cloud computing systems to improve the latency and bandwidth problems cannot satisfy the high computation and fast communication requirements in VR. Edge computing has emerged as a promising solution that can be applied in VR to optimize the latency and bandwidth prob-lems. However, another challenge is applying edge computing technology to improve the seamless for the VR users. Based on this, this paper proposes an edge-end collaboration testbed called Insta VarjoLive and conducts the experiments on the real-time 360° video live streaming seamless using VR headsets. We compared our experiments with the 360° videos watched by users from the cloud through VR headset and obtained the results showing that the edge-assisted 360° videos live streaming method has three major advantages: better real-time delivery, lower response time, and higher bandwidth guaranteed. Furthermore, we tested our experiments and discussed the other possible optimization methods in the future.

Virtual reality applications for industrial training have widespread benefits for simulating various scenarios and conditions. We present an empirical evaluation of VR training approach using kinesthetics learning strategy in industrial maintenance training, specifically the hydraulic manufacturing industry. Through our collaboration with a leading industry partner, a remote multi-user training platform using head-mounted display was designed and implemented. We present the evaluation of the platform with two diverse cohorts of novice users and industry contractors, in comparison to traditional training using slides, photos, and videos. The results show that VR training is engaging and effective in boosting trainee’s confidence, especially for novice users. Our studies highlight the impact of virtual reality training on trainee experience, performance, and skills transfer, with reflections on the differences between novice and industry trainees.

Human emotion recognition has been widely used and intensively studied in many areas such as e-Commerce, online education, healthcare, human-computer interaction and recently human-centric software engineering (HCSE). HCSE investigates human factors in the entire software development lifecycle, and human emotion can be used in many scenarios such as requirement gathering and usability testing. However, even though existing studies have already shown the advantages of emotion recognition with multi-source data such as text, audio and video, the current research and practice in HCSE are primarily based on single-source data. In addition, emotion recognition in HCSE faces several challenges such as multiple participants, changing environments and real-time requirement. To tackle these challenges, this paper proposes Edge4Emotion, a novel edge computing-based multisource human emotion recognition platform for HCSE. Edge4Emotion takes the advantage of the edge computing paradigm to efficiently support the collection of multi-source data such as audio, video and physiological data from various IoT devices, and emotion recognition with both single- and multisource models. As an on-going project, this paper focuses on the platform design and the preliminary evaluation of the platform with representative emotion recognition applications. The platform will be further extended to include more multi-source learning models and serve as an open-source platform for the development and evaluation of multi-source emotion recognition models for HCSE.

While HCI researchers have begun designing personalised VR experiences for older adults, there has been limited research examining the use of social VR - where users interact via avatars in a virtual environment. Avatar-mediated communication (AMC) is a crucial component of the social VR experience, but older users’ experience with AMC is poorly understood. We conducted a five-month study with 16 older adults evaluating a co-designed social VR prototype. Results show that AMC in social VR was seen as medium that supported introverted users to express themselves and was viewed as offering advantages when discussing sensitive topics. Our study provides new insights into how older adults view AMC in social VR as a communication medium and we contribute six design reflections, based on our results, that highlight the steps that can be taken to ensure that AMC in social VR can meet the communication needs of older users.

Recognition of human behaviours including body motions and facial expressions plays a significant role in human-centric software engineering. However, due to the data and computation intensive nature of human behaviour recognition through video analytics, expensive powerful machines are often required, which could hinder the research and application in human-centric software engineering. To address such an issue, this paper proposes a cost-effective human behaviour recognition system named Edge4Real which can be easily deployed in an edge computing environment with commodity machines. Compared with existing centralised solutions, Edge4Real has three major advantages including cost-effectiveness, easy-to-use, and real-time. Specifically, Edge4Real adopts a distributed architecture where components such as motion capturing, human behaviour recognition, data decoding and extraction, and the application of the recognition result, can be deployed on separated end devices and edge nodes in an edge computing environment. Using a virtual reality application which can capture a user's motion and translate into the motion of a 3D avatar in real time, we successfully validate the effectiveness of the system and demonstrate its promising value to the research and application of human-centric software engineering. The demo video can be found at https://youtu.be/tnEshD8j-kA.

Wearable technologies offer potential in supporting assessment of lower limb movements in video consultations, which otherwise are challenging to assess. Yet there remains a limited understanding of how such technologies can be used to improve video consultations in a hospital setting and how they contribute to the clinician-patient interactions over a distance. In this article, we report on the findings of the field evaluation of a wearable technology— SoPhy. SoPhy consists of a pair of sensor embedded socks that capture the lower limb movements of a patient and a web-interface to visualise these movements for the remote physiotherapist. Our study demonstrates that SoPhy helped the physiotherapist in identifying the subtle differences in patients’ movements across all six phases of a consultation. SoPhy increased the confidence of the physiotherapist and guided more accurate assessment of the patients. SoPhy visualisation enhanced the overall clinician-patient communication and offered a better understanding of the therapy goals to the patients. Using the characteristics of the visualisations, patients were able to plan specific goals. We discuss how SoPhy helped in addressing challenges in video consultations experienced by a physiotherapist, and beyond that, how it enabled collective reflection between therapist and patient.

Social virtual reality (social VR) is an emerging technology that has the potential to support unique social experiences for groups of older adults. In this paper we explore the use of social VR to support group reminiscence, an activity that has been shown to have a positive impact on the lives of older adults. We developed School Days, a bespoke social VR application that enables groups of geographically dispersed older adults to meet in a virtual environment to reminisce about their school experiences. We conducted a user study over the course of 5 months with 16 participants aged 70--81 to evaluate how School Days supported reminiscence. In this paper, we focus on how the use of reminiscence scaffolding features in School Days impacted on the older adults' ability to participate more fully in the reminiscence activities. Our results illustrate the value of social VR for connecting older adults over distance, and contribute new knowledge of how virtual environments can be designed to scaffold reminiscence; how techniques such as 3D conversation starters and individual artefacts can be used to scaffold reminiscence; and how pre-recorded holographic stories (Avacasts) can be used to introduce new perspectives and prompt self-reflection. We contribute five design reflections aimed at guiding the design of future reminiscence tools in social VR.

There is growing interest in technologies that allow older adults to socialise across geographic boundaries. An emerging technology in this space is social virtual reality (VR). In this paper we report on a series of participatory design workshops that involved extended in-depth collaboration with 22 older adults (aged 70-81), that aimed to understand their views on the types of social VR experiences that they saw as being of value to older adults. This process culminated in a reminiscence-based social VR concept. Our study identifies: participants' ideas about the types of social VR experiences they found appealing; the potential for social VR as a powerful reminiscence tool; how social VR might be used as a tool to challenge ageing stereotypes and promote healthy ageing. Reflecting on the design process, we discuss how the diverse participant groups and the complexities involved in mediating between designers, the technical team, and participants could inform future design work.

We present two studies to discuss the design, usability analysis, and educational outcome resulting from our system Augmented Body in physiotherapy classroom. We build on prior user-centric design work that investigates existing teaching methods and discuss opportunities for intervention. We present the design and implementation of a hybrid system for physiotherapy education combining an on-body projection based virtual anatomy supplemented by pen-based tablets to create real-time annotations. We conducted a usability evaluation of this system, comparing with projection only and traditional teaching conditions. Finally, we focus on a comparative study to evaluate learning outcome among students in actual classroom settings. Our studies showed increased usage of visual representation techniques in students'11 note taking behavior and statistically significant improvement in some learning aspects. We discuss challenges for designing augmented reality systems for education, including minimizing attention split, addressing text-entry issues, and digital annotations on a moving physical body.

Current techniques for teaching spinal mobilisation follow the traditional classroom approach: an instructor demonstrates a technique and students attempt to emulate it by practising on each other while receiving feedback from the instructor. This paper introduces SpinalLog, a novel tangible user interface (TUI) for teaching and learning spinal mobilisation. The system was co-designed with physiotherapy experts to look and feel like a human spine, supporting the learning of mobilisation techniques through real-time visual feedback and deformation based passive haptic feedback. We evaluated Physical Fidelity, Visual Feedback, and Passive Haptic Feedback in an experiment to understand their effects on physiotherapy students' ability to replicate a mobilisation pattern recorded by an expert. We found that simultaneous feedback has the largest effect, followed by passive haptic feedback. The high fidelity of the interface has little effect, but it plays an important role in the perception of the system's benefit.

A growing body of research is examining the way that virtual reality (VR) technology might enrich the lives of older adults. However, no studies have yet examined how this technology---combining head mounted displays, motion tracking, avatars, and virtual environments---might contribute to older adult wellbeing by facilitating greater social participation (social VR). To address this gap, we conducted three workshops in which 25 older adults aged 70 to 81 explored the utility of social VR as a medium for communicating with other older adults. Participants first created embodied avatars that were controlled through natural gestures, and subsequently used these avatars in two high-fidelity social VR prototypes. Findings from the workshops provide insight into older adults' design motivations when creating embodied avatars for social VR; their acceptance of social VR as a communication tool; and their views on how social VR might play a beneficial role in their lives. Outcomes from the workshops also illustrate the critical importance our participants placed onbehavioural anthropomorphism ---the embodied avatars' ability to speak, move, and act in a human-like manner--- alongsidetranslational factors, which encapsulate issues relating to the way physical movements are mapped to the embodied avatar and the way in which errors in these mappings may invoke ageing stereotypes. Findings demonstrate the critical role that these characteristics might play in the success of future social VR applications targeting older users. We translate our findings into a set of design considerations for developing social VR systems for older adults, and we reflect on how our participants' experiences can inform future research on social virtual reality.

Human body in HCI is often seen as an actuator for issuing commands and providing input to digital systems. We present the concept of the body as a canvas, in which the body acts as both an actuator and a display for information. Body as a canvas creates an interaction loop where interaction with information causes changes in the body, which in turn changes the display of information. Our qualitative study using an on-body projection system in a public exhibition investigates this concept with regards to body characteristics, types of body input, interactions between multiple bodies, and comparison to other display technologies. Our findings show that body as a canvas create connectedness between the body and information. Finally, we discuss how body characteristics and appearances can complement the information, when the body acts as a canvas.

Physiotherapists are increasingly using video conferencing tools for their teleconsultations. Yet, the assessment of subtle differences in body movements remains a challenge. To support lower limb assessment in video consultations, we present SoPhy, a wearable technology consisting of a pair of socks with embedded sensors for patients to wear; and a web interface that displays information about range of weight distribution, foot movement, and foot orientation for physiotherapists in real-time. We conducted a laboratory study of 40 video consultations, in which postgraduate physiotherapy students assessed lower limb function. We compare assessment with and without SoPhy. Findings show that SoPhy increased the confidence in assessing squats exercise and fewer repetitions were required to assess patients when using SoPhy. We discuss the significance of SoPhy to address the challenges of assessing bodily information over video, and present considerations for its integration with clinical practices and tools.

Physiotherapy students often struggle to translate anatomical knowledge from textbooks into a dynamic understanding of the mechanics of body movements in real life patients. We present the Augmented Studio, an augmented reality system that uses body tracking to project anatomical structures and annotations over moving bodies for physiotherapy education. Through a user and learner centered design approach, we established an understanding that through augmentation and annotation, augmented reality technology can enhance physiotherapy education. Augmented Studio enables augmentation through projection mapping to display anatomical information such as muscles and skeleton in real time on the body as it moves. We created a technique for annotation to create projected hand-drawing on the moving body, to enable explicit communication of the teacher's clinical reasoning strategies to the students. Findings from our pilot usability study demonstrate a more engaging learning and teaching experience and increased communication between teacher and students when using Augmented Studio.

This paper investigates the challenges of bodily communication during video-based clinical consultations. While previous works describe the lack of eye contact and gestures over video, it is unclear how these limitations impact the course of a clinical consultation, particularly in a domain like physiotherapy where the focus is on improving body movements and functioning. To contribute to this understanding, we conducted observations of 10 naturally occurring video and face-to-face consultations for physiotherapy. We found that clinicians rely on a variety of incidental bodily cues and fine-details of body movements to assess and examine the patient. These bodily cues were noticeable during face-to-face consultations; however, a variety of bodily cues got missed over video. Consequently, video consultations became conversational where the clinicians used verbal conduct to get a fair understanding of the patient's health. To guide design of future video consultation systems, we reflect on our understanding as 4 design sensitivities: Visual Acuity, Field-of-view, Clinical Asymmetries, and Time Sequence.

We present a passive deformable haptic (PDH) glove to enhance mobile immersive augmented reality manipulation with a sense of computer-captured touch, responding to objects in the physical environment. We extend our existing pinch glove design with a Digital Foam sensor, placed under the palm of the hand. The novel glove input device supports a range of touch-activated, precise, direct manipulation modeling techniques with tactile feedback including hole-punching, trench cutting, and chamfer creation. The PDH glove helps improve a user's task performance time, decrease error rate and erroneous hand movements, and reduce fatigue.

This paper explores enhancing mobile immersive augmented reality manipulations by providing a sense of computer-captured touch through the use of a passive deformable haptic glove that responds to objects in the physical environment. The glove extends our existing pinch glove design with a Digital Foam sensor that is placed under the palm of the hand. The novel glove input device supports a range of touch-activated, precise, direct manipulation modeling techniques with tactile feedback including hole cutting, trench cutting, and chamfer creation. A user evaluation study comparing an image plane approach to our passive deformable haptic glove showed that the glove improves a user's task performance time, decreases error rate and erroneous hand movements, and reduces fatigue.

We present a set of distance-based interaction techniques for modeling and manipulation, enabled by a new input device called the ultrasonic gloves. The ultrasonic gloves are built upon the original design of the pinch glove device for virtual reality systems with a tilt sensor and a pair of ultrasonic transducers in the palms of the gloves. The transducers are distance-ranging sensors that allow the user to specify a range of distances by natural gestures such as facing the palms towards each other or towards other surfaces. The user is able to create virtual models of physical objects by specifying their dimensions with hand gestures. We combine the reported distance with the tilt orientation data to construct virtual models. We also map the distance data to create a set of affine transformation techniques, including relative and fixed scaling, translation, and rotation. Our techniques can be generalized to different sensor technologies.

We present a set of techniques to perform in-situ refinements on simple geo-referenced models using mobile augmented reality systems. The refinements include affine transformations to the model and surface feature additions, including high detail concave and convex features. The techniques employ pinch gloves and a single-point laser rangefinder augmented with an orientation sensor as input devices. Finished models can be exported for use with other geospatial applications. The proposed techniques are intended to be an effective and elegant approach to enhancing outdoor models using mobile augmented reality.

We present a design and implementation of an in-situation menu system for loading and visualising 3D models in a physical world context. The menu system uses 3D objects as menu items, and the whole menu is placed within the context of the augmented environment. The use of 3D objects supports the visualisation and placement of 3D models into the augmented world. The menu system employs techniques for the placement of 3D models in two relative coordinate systems: head relative and world relative.