Due to age-related cognitive and physical decline, older adults face numerous difficulties when learning new functions of smartphone applications. However, older adults often struggle to ask questions clearly and follow instructions independently. Through a formative study (N=16), we identified the behaviors and challenges of older adults seeking help independently and analyzed the effective mechanism of in-person instruction. Based on these findings, we proposed GuideMe, an in-situ conversational instruction system for older adults’ application learning. GuideMe utilizes Vision-Language-Models to analyze multimodal context in users’ situations, then assists users in confirming their intentions by asking clarifying questions, and finally provides step-by-step instructions using in-situ highlight and deictic gestures. We conducted a user study (N=18) that demonstrated that GuideMe significantly reduced users’ cognitive load during learning, helped them ask questions and follow instructions efficiently, and achieved performance comparable to that of in-person instruction.

The “Ideorealm Alignment of Paintings and Poems (IA-PP)” theory rooted in Chinese classical aesthetics offers a perspective for exploring poetry’s deep connotations. This study presents PoemPalette, a novel IA-PP creative-exploration tool that integrates Generative Artificial Intelligence (GAI) to guide poetry enthusiasts in actively constructing an ideorealm for the poetic painting they envision, informed by a formative study with six experts. We extract the core symbols of poetry, transform them into Scene Graph (SG), and generate images for users to freely compose, enabling IA-PP creative exploration. The system incorporates Large Language Model (LLM) agents to enhance the foundational understanding of poetry. In a controlled experiment on Chinese poetry and Japanese haiku with 60 participants, we analyze which interaction mechanisms most contribute to foundational understanding and creative outcomes, compared with both AI and non-AI baselines. Situated within East Asian poetry traditions, this study introduces cultural theories to guide the design of AI co-creation tools, using a graph-based interface of interpretable intermediate representations.

Ink painting, renowned for its aesthetics and historical significance, plays a vital role in global art. Further, 3D ink art extends this tradition into spatial forms, enriching digital media like animation and games. However, existing methods for 3D ink creation demand expertise in both 3D modeling and ink aesthetics, limiting novice participation and 3D ink application. Through formative research with four experts, including ink painting artists and 3D designers, we summarize the core challenge: how to preserve the expressive pattern of ink paintings while constructing 3D structures. To tackle this challenge, we introduce 3DInkGen, a system transforming 2D ink elements into editable 3D compositions. 3DInkGen follows a four-stage workflow: element extraction, form generation, 3D reconstruction, and style transfer. A user study with sixteen novices showed 3DInkGen lowers technical barriers and enables intuitive 3D composition. The four experts believe novice-created works captured the artistic style of ink painting while maintain 3D structure of elements.

Recent advances in artificial intelligence-generated content (AIGC) have led to the rapid generation of high-quality images. AIGC has attracted the attention of commercial digital art asset creators. Traditional artist-led processes contrast with current AI tools that often reduce creators to passive roles. This study examines the integration of AI image generation into commercial digital art, emphasizing the importance of preserving creators’ creative autonomy. Our formative study (S1) involved interviews with commercial digital art creators, highlighting a need for greater control and transparency in AI-assisted painting. In response, we developed RealtimeGen, an integrated tool that merges human creativity with AI’s capabilities, allowing creators to intervene in the generative process. A user study (S2) comparing RealtimeGen with the popular AIGC tool Stable Diffusion was also carried out. The results showed its enhanced user experience and workflow compatibility. Our work contributes to understanding and improving AI-assisted painting workflows for commercial creators, offering them greater creative agency.