We investigate retargeted sketching, a concept for leveraging tangible interactions for sketching on 3D virtual objects. Retargeted sketching enables users to draw curves on virtual objects using physical proxies while compensating for perceptual disparities between physical and virtual spaces. To study this concept in a systematic manner, we implemented an interface with a spherical tangible proxy and a virtual deformed sphere, along with two additional interfaces using a haptic stylus to gain insights into how kinesthetic disparity affects the efficacy of sketching. Our analysis of average deviation, completion time, and smoothness shows that tangible interactions outperform haptics interfaces in terms of smoothness and completion time with marginal reduction in accuracy.

In this work, we investigate the role of generative AI agents as reflective partners in engineering design. While such models are increasingly used to generate design solutions, concerns remain about their potential to diminish designers’ critical thinking and reasoning skills. To address this, we develop a mixed-initiative conversational framework that positions large language and vision–language models as reflective thinking partners rather than solution providers. The framework is structured around five contextual information channels, namely task–role context, design representations, historical context, evaluation signals, and target references, that enable AI agents to ask reflective questions and provide explanations and suggestions. To study this framework within a concrete design context, we develop an interactive tool that embodies the notion of contextual fidelity of 2D structure design tasks. We implement varying levels of contextual fidelity, defined by the extent of contextual information available to the agent. We evaluate these levels through a between-subjects study with forty-six participants, comparing a high-fidelity and a low-fidelity agent against a control group without AI support. We examine the impact of the agents on how users think, talk and act, using a comprehensive set of metrics, including coarse-level design objectives (deformation and material usage), solution quality metrics (structural and geometric analysis), process-oriented measures (design space exploration patterns and trajectories, design strategy shifts), conversational dynamics (thematic and temporal analysis), and subjective surveys (NASA-TLX, Cognitive Load Theory, Trust in AI). Our analyses show that while conversational agents do not immediately help improve coarse-level design objectives, they significantly shape nuanced aspects of design processes and outcomes. Interaction with the agents critically influences how users explore the design space, where agent-supported groups exhibited more focused exploration patterns compared to the control group’s broader trial-and-error approaches. Furthermore, interactions with the high-fidelity agent led to solutions with higher symmetry and topological alignment with optimal designs, fostered deeper reflection, reduced mental demand, and supported more deliberate design decisions. Building on these findings, we discuss broader implications of AI agents for problem-solving processes and outline guidelines for designing adaptive and generalizable frameworks for different domains.

In this paper, we investigate blended physical-digital kinesthetic feedback (or blended haptics in short) as a means for controlled three-dimensional design ideation in mixed reality (MR) environments. We define blended haptics as a spatial interaction wherein a physical object (e.g. a 3D printed human head) provides a specific design context for a user to generate ideas through physical manipulation of and on the object (e.g. drawing a digital sketch of a helmet) in the virtual environment. Using 3D wire-frame modeling as a concrete digital prototyping context, we investigate this idea of blended haptics in terms of how it supports design cognition specifically in spatial user interfaces. For this, we implemented a modeling tool as an experimental setup that allows a user to directly create curve-networks (wire-frame models) on a physical object (i.e. a contextual proxy) with one hand while simultaneously controlling the object with the other hand using a tracked turn-table. The key idea is that the user can simultaneously experience kinesthetic feedback from both the physical objects as well as the digital wire-frame models. To systematically investigate our approach, we conducted a comparative user evaluation of conceptual design tasks performed by two groups of users, one with the blended haptics (e.g. physical head and digital helmet) and the other with purely digital haptic feedback (e.g. digital head and digital helmet). Our study shows that blended haptics required less physical effort in the design task and resulted in concepts with higher novelty score as compared to using purely digital haptic feedback.

Augmented Reality (AR) training is a cost-effective and safe alternative to traditional instructional methods. However, training novices in basic mid-air AR interactions remains challenging. To address this, we aimed to: (a) develop a robust metric to evaluate user performance across different AR interaction techniques and develop adaptation models to predict additional training requirements; (b) evaluate the adaptation models using a neuroergonomics approach. We conduct a two-phase study during which, novice participants perform simple AR interactions: poking and raycasting. In Phase-I, twenty-seven participants’ data is used to identify a bi-variate performance metric based on median completion ime and consistency. Unsupervised models are trained using this metric to classify participants as low/high performers. In Phase-II, we evaluate the models on twenty-one new participants and analyze the differences in performance, neural activity and heart-rate variability between low/high performers. Our study showcases the effectiveness of our models and further discusses the potential of integrating neuroergonomics for advanced AR-based training applications.

This Research Work In Progress Paper examines empirical evidence on the impacts of feedback from an intelligent tutoring software on sketching skill development. Sketching is a vital skill for engineering design, but sketching is only taught limitedly in engineering education. Teaching sketching usually involves one-on-one feedback which limits its application in large classrooms. To meet the demands of feedback for sketching instruction, SketchTivity was developed as an intelligent tutoring software. SketchTivity provides immediate personalized feedback on sketching freehand practice. The current study examines the effectiveness of the feedback of SketchTivity by comparing students practicing with the feedback and without. Students were evaluated on their motivation for practicing sketching, the development of their skills, and their perceptions of the software. This work in progress paper examines preliminary analysis in all three of these areas.

In this article, we introduce a framework for the geometric design and fabrication of a family of geometrically interlocking space-filling shapes, which we call woven tiles. Our framework is based on a unique combination of (1) Voronoi partitioning of space using curve segments as the Voronoi sites and (2) the design of these curve segments based on weave patterns closed under symmetry operations. The underlying weave geometry provides an interlocking property to the tiles and the closure property under symmetry operations ensure single tile can fill space. In order to demonstrate this general framework, we focus on specific symmetry operations induced by fabric weaving patterns. We specifically showcase the design and fabrication of woven tiles on flat and curved domains by using the most common 2-fold fabrics, namely, plain, twill, and satin weaves. We further evaluate and compare the mechanical behavior of the so created woven tiles through finite element analysis.

We introduce a novel workflow, QCue, for providing textual stimulation during mind-mapping. Mind-mapping is a powerful tool whose intent is to allow one to externalize ideas and their relationships surrounding a central problem. The key challenge in mind-mapping is the difficulty in balancing the exploration of different aspects of the problem (breadth) with a detailed exploration of each of those aspects (depth). Our idea behind QCue is based on two mechanisms: (1) computer-generated automatic cues to stimulate the user to explore the breadth of topics based on the temporal and topological evolution of a mind-map and (2) user-elicited queries for helping the user explore the depth for a given topic. We present a two-phase study wherein the first phase provided insights that led to the development of our work-flow for stimulating the user through cues and queries. In the second phase, we present a between-subjects evaluation comparing QCue with a digital mind-mapping work-flow without computer intervention. Finally, we present an expert rater evaluation of the mind-maps created by users in conjunction with user feedback.

In this paper, we introduce a geometric design and fabrication framework for a family of interlocking space-filling shapes which we call bi-axial woven tiles. Our framework is based on a unique combination of (1) Voronoi partitioning of space using curve segments as the Voronoi sites and (2) the design of these curve segments based on weave patterns closed under symmetry operations. The underlying weave geometry provides an interlocking property to the tiles and the closure property under symmetry operations ensure single tile can fill space. In order to demonstrate this general framework, we focus on specific symmetry operations induced by bi-axial weaving patterns. We specifically showcase the design and fabrication of woven tiles by using the most common 2-fold fabrics called 2-way genus-1 fabrics, namely, plain, twill, and satin weaves.

Drawing curves in mid-air with fingers is a fundamental task with applications to 3D sketching, geometric modeling, handwriting recognition, and authentication. Mid-air curve input is most commonly accomplished through explicit user input; akin to click-and-drag, the user may use a hand posture (e.g. pinch) or a button-press on an instrumented controller to express the intention to start and stop sketching. In this paper, we present a novel approach to recognize the user's intention to draw or not to draw in a mid-air sketching task without the use of postures or controllers. For every new point recorded in the user's finger trajectory, the idea is to simply classify this point as either hover or stroke. Our work is motivated by a behavioral study that demonstrates the need for such an approach due to the lack of robustness and intuitiveness while using hand postures and instrumented devices. We captured sketch data from users using a haptics device and trained multiple binary classifiers using feature vectors based on the local geometric and motion profile of the trajectory. We present a systematic comparison of these classifiers and discuss the advantages of our approach to spatial curve input applications.

We present Co-3Deator, a sketch-based collaborative 3D modeling system based on the notion of "team-first" ideation tools, where the needs and processes of the entire design team come before that of an individual designer. Co-3Deator includes two specific team-first features: a concept component hierarchy which provides a design representation suitable for multi-level sharing and reusing of design information, and a collaborative design explorer for storing, viewing, and accessing hierarchical design data during collaborative design activities. We conduct two controlled user studies, one with individual designers to elicit the form and functionality of the collaborative design explorer, and the other with design teams to evaluate the utility of the concept component hierarchy and design explorer towards collaborative design ideation. Our results support our rationale for both of the proposed team-first collaboration mechanisms and suggest further ways to streamline collaborative design.

We present, Window-Shaping, a tangible mixed-reality (MR) interaction metaphor for design ideation that allows for the direct creation of 3D shapes on and around physical objects. Using the sketch-and-inflate scheme, our metaphor enables quick design of dimensionally consistent and visually coherent 3D models by borrowing visual and dimensional attributes from existing physical objects without the need for 3D reconstruction or fiducial markers. Through a preliminary evaluation of our prototype application we demonstrate the expressiveness provided by our design workflow, the effectiveness of our interaction scheme, and the potential of our metaphor.

In this paper, we explore quick 3D shape composition during early-phase spatial design ideation. Our approach is to re-purpose a smartphone as a hand-held reference plane for creating, modifying, and manipulating 3D sweep surfaces. We implemented MobiSweep, a prototype application to explore a new design space of constrained spatial interactions that combine direct orientation control with indirect position control via well-established multi-touch gestures. MobiSweep leverages kinesthetically aware interactions for the creation of a sweep surface without explicit position tracking. The design concepts generated by users, in conjunction with their feedback, demonstrate the potential of such interactions in enabling spatial ideation.

Multitouch surfaces are becoming prevalent, but most existing technologies are only capable of detecting the user's actual points of contact on the surface and not the identity, posture, and handedness of the user. In this paper, we define the concept of extended multitouch interaction as a richer input modality that includes all of this information. We further present a practical solution to achieve this on tabletop displays based on mounting a single commodity depth camera above a horizontal surface. This will enable us to not only detect when the surface is being touched, but also recover the user's exact finger and hand posture, as well as distinguish between different users and their handedness. We validate our approach using two user studies, and deploy the technique in a scratchpad tool and in a pen + touch sketch tool.