Joon Gi Shin

dblp:213/8309 · also Joongi Shin · DBLP profile ↗
← Back
11ranked-venue papers
5as first author
6since 2021 · last 2026
0000-0002-9974-8283ORCID · verified

Domains — the database's venue-derived domains; a paper can count in several

Human-computer interaction and ubiquitous computing · 10 · 5 first-author · 6 since 2021Graphics, computer vision, multimedia, augmented reality and games · 1
YearPublicationVenuePosition
2026 Understanding Spatiotemporal-Aware Multimodal Conversational Search in the Outdoor Urban Space
abstract
Emerging multimodal conversational search (MCS) tools (e.g., Gemini Live) allow users to search for spatiotemporal information through natural language dialogues as they move through urban space. Despite the growing popularity of these tools, there is limited understanding of how people engage with this technology. To address this gap, we developed UrbanSearch, an MCS technology probe designed to capture the user’s current geolocation, time, and visual surroundings. A contextual inquiry (N=23) revealed that MCS tools provide two core values: requiring low effort in forming queries while offering highly relevant responses, and functioning as a central information gateway. As a promising technology, MCS supports environmental learning, in-situ decision making, and personalized navigation. Participants also revealed unmet needs for spatial reasoning and transparent integration of multi-source information, along with concerns related to peripheral awareness, social context, and personal space. Drawing from the findings, we discuss design implications for future MCS tools in urban spaces.
Jiangnan Xu, Suyeon Seo, Joni Salminen, Michael Saker, Joon Gi Shin, Alan Chamberlain, Konstantinos Papangelis, Daehyun Kim 0005
CHI5
2025 Infinity Book: Speculating Literary Expressions in the Age of Generative AI
abstract
This paper explores the ethical, social, political, and philosophical implications of generative AI (GenAI) on human creativity, contributing to the current discussions on the impact of AI. We use speculative and critical design as our approach to avoid abstract guesswork and provide more nuanced and concrete insights into the matter at hand. We conducted five speculative design workshops centered on Infinity Book, a fictional system capable of generating any kind of literary work. Participants used brainwriting, the Future Ripples method, and dialogue-labs to explore potential futures and socio-material impacts of such technology. Reflexive Thematic Analysis was employed to analyze the results from the workshops, and we developed twelve design fictions that illustrate diverse uses and societal implications of GenAI. Based on these results, we formulated three strong concepts, Authenticity, Creative Agency, and Liveness, that link concrete design considerations with broader philosophical discussions. © 2025 Copyright held by the owner/author(s).
Jussi Holopainen, Joon Gi Shin, Erkka Nissinen, Andrés Lucero
Conference on Designing Interactive Systems3
2025 No Evidence for LLMs Being Useful in Problem Reframing
abstract
Problem reframing is a designerly activity wherein alternative perspectives are created to recast what a stated design problem is about. Generating alternative problem frames is challenging because it requires devising novel and useful perspectives that fit the given problem context. Large language models (LLMs) could assist this activity via their generative capability. However, it is not clear whether they can help designers produce high-quality frames. Therefore, we asked if there are benefits to working with LLMs. To this end, we compared three ways of using LLMs (N=280): 1) free-form, 2) direct generation, and 3) a structured approach informed by a theory of reframing. We found that using LLMs does not help improve the quality of problem frames. In fact, it increases the competence gap between experienced and inexperienced designers. Also, inexperienced ones perceived lower agency when working with LLMs. We conclude that there is no benefit to using LLMs in problem reframing and discuss possible factors for this lack of effect.
Joon Gi Shin, Anna Polyanskaya, Andrés Lucero, Antti Oulasvirta
CHI1
2025 Generative AI personas considered harmful? Putting forth twenty challenges of algorithmic user representation in human-computer interaction
abstract
• Shows how GenAI fundamentally transforms existing persona development issues through evolutionary amplification rather than creating entirely new problems, with traditional biases becoming algorithmic discrimination and manual inconsistencies becoming convincing AI hallucinations. • Reveals how traditional limitations manifest differently in GenAI contexts across transparency, fairness, reliability, and control domains, with expert validation showing 60% of challenges are more problematic for GenAIPs than conventional approaches. • Documents how GenAI transforms not just technical challenges but harm distribution, with persona developers facing operational complexity while target user groups bear severe consequences through systematic misrepresentation and exclusion. • Provides evidence that while GenAIPs appear to solve traditional limitations, they transform existing challenges into more complex forms requiring novel validation approaches and human-AI collaboration frameworks for responsible implementation. Generative AI personas (GenAIPs) promise user-centred design efficiency, but their impact on different persona challenges remains unexplored. Inspired by Dijkstra’s classic essay on harmful programming constructs, we analyze twenty challenges in persona development using Human-Centered AI principles. Through literature review and expert survey (n=17), we find that GenAIPs transform rather than eliminate traditional persona challenges. Experts rated all challenges as problematic for GenAIPs (M > 4.0), with the highest concerns for hallucinations (M=5.94), over-sanitization (M=5.82), and lack of standardization (M=5.59). 12 out of 20 challenges are considered more problematic for GenAIPs than conventional personas, particularly bias amplification, validation challenges, and accessibility without expertise. We provide HCAI-grounded guidelines demonstrating that effective GenAIP implementation requires human-AI collaboration rather than automation and prioritizing user welfare over technical efficiency.
Danial Amin 0001, Joni Salminen, Jim Jansen, Joon Gi Shin, Daehyun Kim 0005
Int. J. Hum. Comput. Stud.4
2024 Understanding Human-AI Workflows for Generating Personas
abstract
One barrier to deeper adoption of user-research methods is the amount of labor required to create high-quality representations of collected data. Trained user researchers need to analyze datasets and produce informative summaries pertaining to the original data. While Large Language Models (LLMs) could assist in generating summaries, they are known to hallucinate and produce biased responses. In this paper, we study human–AI workflows that differently delegate subtasks in user research between human experts and LLMs. Studying persona generation as our case, we found that LLMs are not good at capturing key characteristics of user data on their own. Better results are achieved when we leverage human skill in grouping user data by their key characteristics and exploit LLMs for summarizing pre-grouped data into personas. Personas generated via this collaborative approach can be more representative and empathy-evoking than ones generated by human experts or LLMs alone. We also found that LLMs could mimic generated personas and enable interaction with personas, thereby helping user researchers empathize with them. We conclude that LLMs, by facilitating the analysis of user data, may promote widespread application of qualitative methods in user research.
Joon Gi Shin, Michael A. Hedderich, Bartlomiej Jakub Rey, Andrés Lucero, Antti Oulasvirta
Conference on Designing Interactive Systems1
2022 Chatbots Facilitating Consensus-Building in Asynchronous Co-Design
abstract
Consensus-building is an essential process for the success of co-design projects. To build consensus, stakeholders need to discuss conflicting needs and viewpoints, converge their ideas toward shared interests, and grow their willingness to commit to group decisions. However, managing group discussions is challenging in large co-design projects with multiple stakeholders. In this paper, we investigate the interaction design of a chatbot that can mediate consensus-building conversationally. By interacting with individual stakeholders, the chatbot collects ideas to satisfy conflicting needs and engages stakeholders to consider others’ viewpoints, without having stakeholders directly interact with each other. Results from an empirical study in an educational setting (N = 12) suggest that the approach can increase stakeholders’ commitment to group decisions and maintain the effect even on the group decisions that conflict with personal interests. We conclude that chatbots can facilitate consensus-building in small-to-medium-sized projects, but more work is needed to scale up to larger projects.
Joon Gi Shin, Michael A. Hedderich, Andrés Lucero, Antti Oulasvirta
UIST1
2020 Body Follows Eye: Unobtrusive Posture Manipulation Through a Dynamic Content Position in Virtual Reality
abstract
While virtual objects are likely to be a part of future interfaces, we lack knowledge of how the dynamic position of virtual objects influences users' posture. In this study, we investigated users' posture change following the unobtrusive and swift motions of a content window in virtual reality (VR). In two perception studies, we estimated the perception threshold on undetectable slow motions and displacement during an eye blink. In a formative study, we compared users' performance, posture change as well as subjective responses on unobtrusive, swift, and no motions. Based on the result, we designed concept applications and explored potential design space of moving virtual content for unobtrusive posture change. With our study, we discuss the interfaces that control users and the initial design guidelines of unobtrusive posture manipulation.
Joon Gi Shin, Doheon Kim, Chaehan So, Daniel Saakes
CHI1
2019 Slow Robots for Unobtrusive Posture Correction
abstract
Prolonged static and unbalanced sitting postures during computer usage contribute to musculoskeletal discomfort. In this paper, we investigated the use of a very slow moving monitor for unobtrusive posture correction. In a first study, we identified display velocities below the perception threshold and observed how users (without being aware) responded by gradually following the monitor's motion. From the result, we designed a robotic monitor that moves imperceptible to counterbalance unbalanced sitting postures and induces posture correction. In an evaluation study (n=12), we had participants work for four hours without and with our prototype (8 in total). Results showed that actuation increased the frequency of non-disruptive swift posture corrections and significantly reduced the duration of unbalanced sitting. Most users appreciated the monitor correcting their posture and reported less physical fatigue. With slow robots, we make the first step toward using actuated objects for unobtrusive behavioral changes.
Joon Gi Shin, Eiji Onchi, Maria Jose Reyes, JunBong Song, Uichin Lee, Seung-Hee Lee, Daniel Saakes
CHI1
2019 Towards large scale high fidelity collaborative augmented reality
Damien Constantine Rompapas, Christian Sandor, Alexander Plopski, Daniel Saakes, Joon Gi Shin, Takafumi Taketomi, Hirokazu Kato 0001
Comput. Graph.5
2018 Interactive and Situated Guidelines to Help Users Design a Personal Desk that Fits Their Bodies
abstract
In this paper, we explored the application of human factor guidelines in personal fabrication. This is useful for several Do-It-Yourself (DIY) scenarios, including users adjusting workstation configurations or designing a desk to fit a single person. We identified a dependency map between the user's anthropometrics, ergonomic pose recommendations, and design dimensions. Based on this, we developed situated and interactive guidelines to assist users in design applications. We applied these guidelines in a Virtual Reality (VR) system that lets users customize their desk and provides real-time feedback and feedforward on pose and design. We evaluated the system with six participants, had each one design a personal desk, fabricated their desks, and let them work on their desks for four hours. The design and evaluation contribute to fabrication tools as it helped users be aware of their pose and ergonomic knowledge, and design for their bodies and needs.
Bokyung Lee, Joon Gi Shin, Hyoshin Bae, Daniel Saakes
Conference on Designing Interactive Systems2
2018 Situated Game Level Editing in Augmented Reality
abstract
Level editors let end-users create custom levels and content within a given video game. In this paper, we explore the concept and design of Augmented reality game level editors. These new types of editors are not only spatial and embodied, but also situated, as they enable users to tailor games to the unique characteristics and emotional value of their own space.
Gary Ng, Joon Gi Shin, Alexander Plopski, Christian Sandor, Daniel Saakes
TEI2