The transition from conceptual sketch to 3D model poses a major challenge in engineering design. While Mixed Reality (MR) enables direct 3D creation, designers face two key challenges: the Initiation Challenge (lack of meaningful starting points) and the Expression Challenge (difficulty in precise mid-air sketching). We introduce GenerativeSurface, leveraging AI to translate natural language descriptions into interactive 3D surfaces that aid designers in 3D sketching, simultaneously addressing both challenges. Our system transforms the design workflow into a fluid "describe, generate, sketch, iterate" cycle. Comparative evaluation against primitive scaffolding and freehand mid-air sketching demonstrates that our approach effectively bridges the gap between designers' conceptual intent and 3D sketch expression in immersive environments.

In this research, we investigate the effectiveness of asymmetric interactions (HandStylus, HandController, and TwoHands) in Augmented Reality (AR), Virtual Reality (VR), and Extended Reality (XR) for 3D digital drawing overlaying on physical and virtual objects. We evaluate the input accuracy and fatigue of these object-based 3D drawing experiences using quantitative measurements and further explore the correlation between these outcomes with subjective questionnaires. We found significant independence between environments and interface types, which considerably influence the performance and usability of 3D immersive drawing. We noted discrepancies between users’ subjective experiences and objective performance. Specifically, although AR drawing on physical objects provides superior accuracy and minimal muscle fatigue due to tangible feedback, and the TwoHands interaction offers the highest precision, the subjective results show the reverse outcome. Based on these findings, we propose design recommendations and discuss directions for future research in immersive drawing environments.

This research investigates fast and precise Virtual Reality (VR) sketching methods with different tool-based asymmetric interfaces. In traditional real-world drawing, artists commonly employ an asymmetric interaction system where each hand holds different tools, facilitating diverse and nuanced artistic expressions. However, in virtual reality (VR), users are typically limited to using identical tools in both hands for drawing. To bridge this gap, we aim to introduce specifically designed tools in VR that replicate the varied tool configurations found in the real world. Hence, we developed a VR sketching system supporting three hybrid input techniques using a standard VR controller, a VR stylus, or a data glove. We conducted a formal user study consisting of an internal comparative experiment with four conditions and three tasks to compare three asymmetric input methods with each other and with a traditional symmetric controller-based solution based on questionnaires and performance evaluations. The results showed that in contrast to symmetric dual VR controller interfaces, the asymmetric input with gestures significantly reduced task completion times while maintaining good usability and input accuracy with a low task workload. This shows the value of asymmetric input methods for VR sketching. We also found that the overall user experience could be further improved by optimizing the tracking stability of the data glove and the VR stylus.