Robot-supported interventions for joint attention (JA) in autistic children have shown encouraging outcomes, yet most remain confined to stationary robots in indoor settings, limiting opportunities for skill generalization and broader developmental benefits. We introduce an intervention that employs a quadruped robot dog as a peer-like partner for JA training across both indoor and outdoor environments. In this intervention, the robot dog directs children’s attention to distributed targets in the environment and initiates JA trials. A four-week pre-post exploratory study with six autistic children demonstrated improvements in JA performance and indications of transfer to daily social communication. Spontaneous behaviors such as motor imitation (crawling) and novel social interactions with the robot also emerged, suggesting potential for broader developmental gains. These findings provide initial evidence for the efficacy of mobile robot-supported JA interventions in naturalistic contexts and offer implications for future design.

In science fiction writing, ideation demands both novelty to construct fictional worlds and consistency to maintain internal and temporal logic within those worlds. While large language models (LLMs) are increasingly adopted as co-creators, generated ideas often lack surprise and struggle to maintain consistency. Moreover, current interaction paradigms of human-AI co-creation systems fail to support the fragmented and iterative nature of science fiction ideation. To address these challenges, we introduce Sci-Fi Spark, a human-AI co-creation system to support inspiration and organization in the ideation phase. The system features an Ideation Canvas for visualizing relationships between fragmented ideas, a Novelty Generator that applies computational creativity strategies to produce novel worldbuilding inspirations, and a Consistency Generator to produce context-aware storyline suggestions. A technical evaluation and a user study with writers show that Sci-Fi Spark enhances both novelty and consistency, while enabling iterative co-ideation with LLMs.

Social chatbots are increasingly studied for their benefits in providing companionship and emotional support. These benefits rely on forming human-chatbot relationships that require credible social identity and reciprocal interaction. Memory plays a dual role: it strengthens social identity by enabling the chatbot to remember, and supports reciprocal interaction when memories are disclosed mutually. We present RECALLbot, an LLM-driven social chatbot that constructs agentic memories, including life-like Me Memory and co-constructed We Memory, and adaptively applies reciprocal disclosure strategies with user controls. In a two-week between-subjects study (N = 40), RECALLbot was compared with a baseline system lacking agentic memories and reciprocal disclosure strategies. Results show that RECALLbot enhanced perceptions of the chatbot’s social identity, elicited more frequent and deeper self-disclosures, and fostered greater trust.

Conceptual CAD requires transforming functional requirements into parametric 3D models, yet existing systems have steep learning curves and limit creativity through premature fixation. Generative AI shows promise in producing diverse alternatives, while current methods mainly reconstruct CAD modeling sequences of existing designs, making them unsuitable for early stages where ideas are vague and intent is difficult to express. We present Req2CAD, an interactive system that enables designers to progress from design problems toward conceptual CAD models through functional decomposition, function–structure reasoning, and component-level CAD creation and iteration. Req2CAD introduces a data annotation pipeline that maps functional requirements to the 3D structural design space, a dual-feature CAD representation to support design space exploration and CAD ideation, and a progressive CAD generation method that enables rapid CAD model creation through multi-modal intent expression. A technical evaluation and user study demonstrate the effectiveness of Req2CAD, highlighting its potential for human–AI co-creation.

Online programming communities give novices places to explore computing through user-generated projects, but limited structure can hinder a steadily challenging learning path. Beginners often struggle to interpret key events and relationships in projects, connect them to core concepts, and remix practices. We present CoRemix, a generative-AI community support system that uses visual flowcharts to clarify project logic. CoRemix introduces a prompting pipeline paired with a visual-textual scaffold that guides learners in constructing flowcharts. We further incorporate static project analysis and retrieval-augmented generation (RAG) to raise the precision of large-language-model outputs. In technical evaluations, static analysis and RAG improved response quality. In a user study, CoRemix outperformed a baseline in helping learners understand complex projects, strengthen computing-concept skills, and report better learning experiences within online communities. These gains include clearer event sequencing, improved identification of relationships across sprites and scripts, stronger remix strategies, and higher perceived scaffolding for progressive challenge.

With the increasing demand for personalized health monitoring through wearable devices, there is a growing need for non-prescription ECG diagnosing, especially during physical activities. However, existing abnormal ECG data are typically measured in clinical settings, reflecting heart waveforms in a resting state. Abnormality classification models based on such data often struggle to maintain high performance during physical activities, leading to increased false alarms and a higher incidence of missed detections. Due to the potential risks associated with having patients engage in physical activity, abnormal ECG data captured during exercise is not readily available, further complicating the development of reliable models for active scenarios. To address this issue, we propose GAEA-Net in this study. Our goal is to utilize exercise ECGs from healthy individuals, which are more easily accessible, combined with resting-state abnormal ECGs, to generate activity-enriched ECGs through synthesis. We conduct abnormal classification on five widely used datasets, achieving average improvements of 1.3% in Accuracy, 1.3% in F1-score, 0.9% in AUROC, 1.6% in MCC, and 1.4% in Cohen's Kappa. Furthermore, a clinical Turing test involving seven experienced cardiologists confirms that our synthesized ECGs exhibit high fidelity. In the diagnostic task, the cardiologists achieved comparable accuracy on synthetic and real ECGs (55.7% vs. 54.9%, p = 0.76).

Medical time series are often irregular and face significant missingness, posing challenges for data analysis and clinical decision-making. Existing methods typically adopt a single modeling perspective, either treating series data as sequences or transforming them into image representations for further classification. In this paper, we propose a joint learning framework that incorporates both sequence and image representations. We also design three self-supervised learning strategies to facilitate the fusion of sequence and image representations, capturing a more generalizable joint representation. The results indicate that our approach outperforms seven other state-of-the-art models in three representative real-world clinical datasets. We further validate our approach by simulating two major types of real-world missingness through leave-sensors-out and leave-samples-out techniques. The results demonstrate that our approach is more robust and significantly surpasses other baselines in terms of classification performance.

A design method card deck helps designers understand and provoke thinking by presenting each method in a simple format and allow designers to switch between methods seamlessly by maintaining the same simple format across the deck. However, recent observations have shown designers hesitate to use a card deck due to the lack of support, while other tools have provided identified support with generative AI. Through a formative study, we identified the specific support designers need when applying the design method cards and intentions in integrating generative AI. Accordingly, we developed the intelligent design method card deck, I-Card, which integrates generative AI to provide applicable design methods, design knowledge and data support, and interactive and dynamic support. A user study demonstrates that I-Card improved the design efficiency and applicability by offering personalized guidance, enhanced decision-making with comprehensive data generation and provided more design inspiration via interactive support.

Programming is essential in K-12 education and fosters computational thinking skills. Given the complexity of programming and the advanced skills it requires, previous research has introduced user-friendly tools to support young learners. However, our interviews with six programming educators revealed that current tools often fail to reflect classroom learning objectives, offer flexible guidance, and foster creativity. Therefore, we introduced MindScratch, a multimodal generative AI (GAI)-powered visual programming support tool. MindScratch aims to balance structured classroom activities with free programming creation, supporting students in completing creative programming projects based on teacher-set learning objectives while also providing programming scaffolding. The results indicate that, compared to the baseline, MindScratch more effectively helps students achieve high-quality projects aligned with learning objectives. It also enhances students’ computational thinking and thinking. Overall, we believe that GAI-driven educational tools like MindScratch offer students a focused and engaging learning experience.

Airflow is recognized as an effective method for inducing the illusion of self-motion (vection) and reducing motion sickness in virtual reality. However, the quantitative relationship between virtual motion and the airflow perceived as consistent with it has not been fully explored. To address this gap, this study conducted three experiments. In Experiment 1, we carried out a series of cross-modal matching tasks to establish the relationship between the speed of virtual motion and the airflow speed perceived as consistent with it, revealing a strong linear correlation. In Experiment 2, we introduced the concept of an "Airflow Gradient" to simulate the bodily sensation of curvilinear motion and examined the relationship between the radius and angular velocity of the motion and the difference in airflow speed between the left and right sides. The results indicated a linear relationship between the radius and the left-right airflow speed difference, while the angular velocity showed a near-quadratic pattern, similar to the centripetal acceleration formula. Based on these findings, Experiment 3 developed a dynamic airflow scheme and compared it with constant airflow and no-airflow conditions during locomotion tasks in a complex urban environment. The results demonstrated that dynamic airflow, which ensures consistency between visual and bodily vection, further reduces motion sickness, enhances presence, and provides a more natural and consistent virtual motion experience.

Bio-inspired design (BID) fosters innovations in engineering. Learning BID is crucial for developing multidisciplinary innovation skills of designers and engineers. Current BID education aims to enhance learners’ understanding and analogical reasoning skills. However, it often heavily relies on the teachers’ expertise. When learners pursue independent learning using some educational tools, they face challenges in understanding and reasoning practice within this multidisciplinary field. Additionally, evaluating their learning outcomes comprehensively becomes problematic. Addressing these challenges, we introduce a LLMs-driven BID education method based on a structured ontology and three strategies: enhancing understanding through LLMs-enpowered "learning by asking", assisting reasoning by providing hints and feedback, and assessing learning outcomes through benchmarking against existing BID cases. Implementing the method, we developed BIDTrainer, a BID education tool. User studies indicate that learners using BIDTrainer understood BID knowledge better, reason faster with higher interactivity than the baseline, and BIDTrainer assessed the learning outcomes consistent with experts.

As Computational Thinking (CT) continues to permeate younger age groups in K-12 education, established CT platforms such as Scratch face challenges in catering to these younger learners, particularly those in the elementary school (ages 6-12). Through formative investigation with Scratch experts, we uncover three key obstacles to children’s autonomous Scratch learning: artist’s block in project planning, bounded creativity in asset creation, and inadequate coding guidance during implementation. To address these barriers, we introduce ChatScratch, an AI-augmented system to facilitate autonomous programming learning for young children. ChatScratch employs structured interactive storyboards and visual cues to overcome artist’s block, integrates digital drawing and advanced image generation technologies to elevate creativity, and leverages Scratch-specialized Large Language Models (LLMs) for professional coding guidance. Our study shows that, compared to Scratch, ChatScratch efficiently fosters autonomous programming learning, and contributes to the creation of high-quality, personally meaningful Scratch projects for children.

Children with high-functioning autism (HFA) often face challenges in emotional recognition and expression, leading to emotional distress and social difficulties. Conversational agents developed for HFA children in previous studies show limitations in children's learning effectiveness due to the conversational agents’ inability to dynamically generate personalized and contextual content. Recent advanced generative Artificial Intelligence techniques, with the capability to generate substantial diverse and high-quality texts and visual content, offer an opportunity for personalized assistance in emotional learning for HFA children. Based on the findings of our formative study, we integrated large language models and text-to-image models to develop a tool named EmoEden supporting children with HFA. Over a 22-day study involving six HFA children, it is observed that EmoEden effectively engaged children and improved their emotional recognition and expression abilities. Additionally, we identified the advantages and potential risks of applying generative AI to assist HFA children in emotional learning.

When translating UI design prototypes to code in industry, automatically generating code from design prototypes can expedite the development of applications and GUI iterations. However, in design prototypes without strict design specifications, UI components may be composed of fragmented elements. Grouping these fragmented elements can greatly improve the readability and maintainability of the generated code. Current methods employ a two-stage strategy that introduces hand-crafted rules to group fragmented elements. Unfortunately, the performance of these methods is not satisfying due to visually overlapped and tiny UI elements. In this study, we propose EGFE, a novel method for automatically End-to-end Grouping Fragmented Elements via UI sequence prediction. To facilitate the UI understanding, we innovatively construct a Transformer encoder to model the relationship between the UI elements with multi-modal representation learning. The evaluation on a dataset of 4606 UI prototypes collected from professional UI designers shows that our method outperforms the state-of-the-art baselines in the precision (by 29.75%), recall (by 31.07%), and F1-score (by 30.39%) at edit distance threshold of 4. In addition, we conduct an empirical study to assess the improvement of the generated front-end code. The results demonstrate the effectiveness of our method on a real software engineering application. Our end-to-end fragmented elements grouping method creates opportunities for improving UI-related software engineering tasks.

Rapid creation of novel product appearance designs that align with consumer emotional requirements poses a significant challenge. Text-to-image models, with their excellent image generation capabilities, have demonstrated potential in providing inspiration to designers. However, designers still encounter issues including aligning emotional needs, expressing design intentions, and comprehending generated outcomes in practical applications. To address these challenges, we introduce AutoSpark, an interactive system that integrates Kansei Engineering and generative AI to provide creativity support for designers in creating automobile appearance designs that meet emotional needs. AutoSpark employs a Kansei Engineering engine powered by generative AI and a semantic network to assist designers in emotional need alignment, design intention expression, and prompt crafting. It also facilitates designers’ understanding and iteration of generated results through fine-grained image-image similarity comparisons and text-image relevance assessments. The design-thinking map within its interface aids in managing the design process. Our user study indicates that AutoSpark effectively aids designers in producing designs that are more aligned with emotional needs and of higher quality compared to a baseline system, while also enhancing the designers’ experience in the human-AI co-creation process.

Text entry in virtual reality (VR) often faces challenges in terms of efficiency and task loads. Prior research has explored various solutions, including specialized keyboard layouts, tracked physical devices, and hands-free interaction. Yet, these efforts often fall short of replicating the efficiency of real-world text entry, or introduce additional spatial and device constraints. This study leverages the extensive capabilities of large language models (LLMs) in context perception and text prediction to enhance text entry efficiency by reducing users’ manual keystrokes. Three LLM-assisted text entry methods - Simplified Spelling, Content Prediction, and Keyword-to-Sentence Generation - are introduced, aligning with user cognition and the contextual predictability of English text at word, grammatical structure, and sentence levels. Through user experiments encompassing various text entry tasks on an Oculus-based VR prototype, these methods demonstrate a 16.4%, 49.9%, 43.7% reduction in manual keystrokes, translating to efficiency gains of 21.4%,74.0%, 76.3%, respectively. Importantly, these methods do not increase manual corrections compared to manual typing, while significantly reducing physical, mental, and temporal loads and enhancing overall usability. Long-term observations further reveal users’ strategies for using these LLM-assisted methods, showing that users’ proficiency with the methods can reinforce their positive effects on text entry efficiency.

Divergent thinking is a process in design by exploring multiple possible solutions, is crucial in the early stages of design to break fixation and expand the design ideation. Design-by-Analogy promotes divergent thinking, by studying solutions have solved similar problems and using this knowledge to make inferences and solve problems in new and unfamiliar situations. Bio-inspired design (BID) is a form of design by analogy and its knowledge provides diverse sources for analogy, making BID knowledge as a potential source for divergent thinking. Existing BID database has focused on collecting BID cases and facilitating the retrieval of biological knowledge. Despite its success, applying BID knowledge into divergent thinking still encounters challenge, as the association between source domain and target domain are always limited within a single case. In this work, a novel approach is proposed to support divergent thinking from three subsequent phases: encoding, retrieval and mapping. Specifically, biological knowledge is encoded in a triple form by employing a large language model (LLM) to extract key information from a well-known BID knowledge base. The created triples are implemented in a semantic network to facilitate bidirectional retrieval modes: problem-driven and solution-driven, as well as mapping for divergent thinking. The mapping algorithm calculates the semantic similarity between nodes in the semantic network based on their attributes in three progressive steps by following the paradigm of divergent thinking. The proposed approach is implemented as tool called AskNatureNet,1 which supports divergent thinking by retrieving and mapping knowledge in a visualized interactive semantic network. An ideation case study on evaluating the effectiveness of AskNatureNet shows that our tool is capable of supporting divergent thinking efficiently.

In graphic design, layout is a result of the interaction between the design elements in the foreground and background images. However, prevalent research focuses on enhancing the quality of layout generation algorithms, overlooking the interaction and controllability that are essential for designers when applying these methods in real-world situations. This paper proposes a user-centered layout design system, Iris, which provides designers with an interactive environment to expedite the workflow, and this environment encompasses the features of user-constraint specification, layout generation, custom editing, and final rendering. To satisfy the multiple constraints specified by designers, we introduce a novel generation model, multi-constraint LayoutVQ-VAE, for advancing layout generation under intra- and inter-domain constraints. Qualitative and quantitative experiments on our proposed model indicate that it outperforms or is comparable to prevalent state-of-the-art models in multiple aspects. User studies on Iris further demonstrate that the system significantly enhances design efficiency while achieving human-like layout designs.

Layout is essential for the product listing pages (PLPs) in mobile shopping applications. To clearly convey the information that consumers require and to achieve specific functions, PLPs layouts often have many variations driven by scenarios. In this work, we study the PLPs layout design for different scenarios and propose a design space to guide the large-scale creation of PLPs. We propose LayoutVQ-VAE, a novel model specialized in generating layouts with internal and external constraints. LayoutVQ-VAE differs from previous methods as it learns a discrete latent representation of layout and can model the relationship between layout representation and scenarios without applying heuristics. Experiments on publicly available benchmarks for different layout types validate that our method performs comparably or favorably against the state-of-the-art methods. Case studies show that the proposed approach including the design space and model is effective in producing large-scale high-quality PLPs layouts for mobile shopping platforms.

With the fast-growing graphical user interface (GUI) development workload in the Internet industry, some work attempted to generate maintainable front-end code from GUI screenshots. It can be more suitable for using user interface (UI) design drafts that contain UI metadata. However, fragmented layers inevitably appear in the UI design drafts, which greatly reduces the quality of the generated code. None of the existing automated GUI techniques detects and merges the fragmented layers to improve the accessibility of generated code. In this paper, we propose UI layers merger (UILM), a vision-based method that can automatically detect and merge fragmented layers into UI components. Our UILM contains the merging area detector (MAD) and a layer merging algorithm. The MAD incorporates the boundary prior knowledge to accurately detect the boundaries of UI components. Then, the layer merging algorithm can search for the associated layers within the components’ boundaries and merge them into a whole. We present a dynamic data augmentation approach to boost the performance of MAD. We also construct a large-scale UI dataset for training the MAD and testing the performance of UILM. Experimental results show that the proposed method outperforms the best baseline regarding merging area detection and achieves decent layer merging accuracy. A user study on a real application also confirms the effectiveness of our UILM.