This demo presents a hands-on opportunity for users to explore a new, immersive, node-based platform for creative coding in virtual reality (VR). The Embodied Coding Environment (ECE) is a merged digital/physical workspace where coding tools, spatial representations of code, the scenes that are the output of the code, and user editing histories are co-located in a 3D space. Visual programming languages (VPL) have been used as an alternative to text-based programming since Sketchpad in 1963. However, the ECE transforms visual programming into an immersive experience, allowing users to enter the world where they are actively coding. Beyond immersion, embodied coding allows users to leverage 3D space to organize, annotate, and interact with their code in ways that would not be possible through 2D representation. Our research goals center on understanding how embodied coding can make learning to code more intuitive and elicit heightened engagement through the learning process. Demo visitors will experience embodied coding first hand through simple programming tasks (e.g., spawning objects, using arrays, creating conditional statements, and more) presented in the ECE via a Meta Quest 2 system. Presenters will guide users through the coding activities and discuss future directions. Handouts with brief descriptions of the ECE platform with links to embodiedcode.net will be available. Access to the website will allow users to login and save their projects.

Eight middle- and high-school Computer Science (CS) teachers in San Diego County were interviewed about the major challenges their students commonly encounter in learning computer programming. We identified strategic design opportunities -- that is, challenges and needs that can be addressed in innovative ways through the affordances of Augmented and Virtual Reality (AR/VR). Thematic Analysis of the interviews yielded six thematic clusters: Tools for Learning, Visualization and Representation, Pedagogical Approaches, Classroom Culture, Motivation, and Community Connections. Within the theme of visualization, focal clusters centered on visualizing problem spaces and using metaphors to explain computational concepts, indicating that an AR/VR coding system could help users to represent computational problems by allowing them to build from existing embodied experiences and knowledge. Additionally, codes clustered within the theme of learning tools reflected educators’ preference for web-based IDEs, which involve minimal start-up costs, as well as concern over the degree of transfer in learning between block- and text-based interfaces. Finally, themes related to motivation, community, and pedagogical practices indicated that the design of an AR coding platform should support collaboration, self-expression, and autonomy in learning. It should also foster selfefficacy and learners’ ability to address lived experience and real-world problems through computational means.

A Machine for Living In is a multi-layered digital media artwork using newly available computational and sensing tools to study the home as a site of intimate life. The project has two distinct phases: the construction and inhabitation of a functional smart home system, followed by an exhibition of processed data as a multi-part digital art installation. For ACM Designing Interactive Systems / Creativity and Cognition 2019, A Machine for Living In is installed as a multi-element piece in the gallery space consisting of a sensor narrative video, overhead camera video, and digital audio. All data were originally recorded in the artist's home over a period of weeks, and here are filtered through diverse processes of machine perception. In a process of joint human-machine authorship, this system produces a complex portrait of the home: as a space of language, intimacy, bodily practice, and quotidian narrative.

Rover is a mechatronic imaging device inserted into quotidian space, transforming the sights and sounds of the everyday through its peculiar modes of machine perception. Using computational light field photography and machine listening it creates a kind of cinema following the logic of dreams: suspended but mobile, familiar yet infinitely variable in detail. Rover draws on diverse traditions of robotic exploration, landscape and still life depiction, and audio field recording to create a hybrid between photography and cinema. Rover won the Best Art Paper award at SIGGRAPH 2017. For ACM Designing Interactive Systems / Creativity and Cognition 2019, Rover is installed on the 70 megapixel VRoom display in the black box theater in CalIT2 at UC San Diego. We have developed new functionality where image synthesis is distributed across the panels of the tiled display, visual and sonic trajectories are shaped using the unique hardware and software capabilities at UCSD. The result is an ultra-high resolution audio-visual experience with 4 channel surround sound.