Early mathematical activities often combine symbolic representations of numbers with physical manipulatives (e.g., numicons, dice, cards) that help children represent quantities. However, children with mathematical difficulties often struggle to relate abstract representations (number symbols) to their physical counterparts (manipulatives). In close collaboration with mathematics education experts, we explored how Augmented Reality (AR) can complement existing educational materials rather than replace them. We designed an AR system that augments existing exercises and conducted a pilot study with 10 children solving addition tasks. Our observations suggest that children enjoyed and successfully used the AR augmentations to complete the tasks. However, the AR system also created undesired side-effect such as an overreliance on the virtual augmentations. We discuss design implications for how AR systems can better complement established educational materials and support the learning of calculation-based arithmetic strategies.

Dyscalculia is a specific learning difficulty that affects the ability to understand and work with numbers, often requiring specialized support strategies to develop foundational math skills. The exercises within these support strategies aim to reduce cognitive load by engaging multiple modalities—such as physically experiencing, verbally reasoning, and visually illustrating mathematical concepts. However, most current exercises are paper-based or de- livered through apps detached from the physical world, even when incorporating real-world elements like finger counting or handling currency. This work-in-progress explores how Augmented Reality (AR) can enhance these support strategies by embedding them in dynamic and interactive physical environments. We present a technical design exploration featuring three prototype applications that support students with dyscalculia (grades 1–9) through embodied and situated mathematical training. Our contribution is an early investigation into how AR can reimagine educational strategies by integrating physical manipulatives with virtual augmentation, creating more engaging math experiences for children with dyscalculia.

Prototyping multimodal interaction in XR (Augmented/Virtual Reality) is challenging for non-technical designers. Moreover, existing XR prototyping tools do not support concurrent multimodal interaction and lack an overall view of the interaction design of the experience. We created Mucho , a novel no-coding immersive tool for prototyping interactive multimodal XR experiences. Mucho supports three modes: In recording, designers can demonstrate input examples to populate a timeline representation with events. In playback, designers can create a video prototype based on the timeline while adding system actions at the corresponding timeframes. In live mode, Mucho creates a state machine to test the interactive experience in runtime. We present an evaluation by demonstration to illustrate how Mucho can prototype a diverse set of examples including modalities such as hand gestures, proximity, speech, and gaze. Finally, we discuss the limitations of our approach and outline future directions to support immersive prototyping of multimodal XR experiences.

Research on enabling novice AR/VR developers has emphasized the need to lower the technical barriers to entry. This is often achieved by providing new authoring tools that provide simpler means to implement XR interactions through abstraction. However, novices are then bound by the ceiling of each tool and may not form the correct mental model of how interactions are implemented. We present XRSpotlight, a system that supports novices by curating a list of the XR interactions defined in a Unity scene and presenting them as rules in natural language. Our approach is based on a model abstraction that unifies existing XR toolkit implementations. Using our model, XRSpotlight can find incomplete specifications of interactions, suggest similar interactions, and copy-paste interactions from examples using different toolkits. We assess the validity of our model with professional VR developers and demonstrate that XRSpotlight helps novices understand how XR interactions are implemented in examples and apply this knowledge in their projects.

This paper proposes a solution for supporting end users in configuring Virtual Reality environments by exploiting reusable templates created by experts. We identify the roles participating in the environment development and the means for delegating part of the behaviour definition to the end users. We focus in particular on enabling end users to define the environment behaviour. The solution exploits a taxonomy defining common virtual objects having high-level actions for specifying event-condition-action rules readable as natural language sentences. End users exploit such actions to define the environment behaviour. We report on a proof-of-concept implementation of the proposed approach, on its validation through two different case studies (virtual shop and museum), and on evaluating the approach with expert users.