Nowadays, Mixed Reality (MR) headsets are a game-changer for knowledge work. Unlike stationary monitors, MR headsets allow users to work with large virtual displays anywhere they wear the headset, whether in a professional office, a public setting like a cafe, or a quiet space like a library. This study compares four different layouts (eye level-close, eye level-far, below eye level-close, below eye level-far) of virtual displays regarding feelings of safety, perceived productivity, and social acceptability when working with MR in public. We test which layout is most preferred by users and seek to understand which factors affect users’ layout preferences. The aim is to derive useful insights for designing better MR layouts. A field study in a public library was conducted using a within-subject design. While the participants interact with a layout, they are asked to work on a planning task. The results from a repeated measure ANOVA show a statistically significant effect on productivity but not on safety and social acceptability. Additionally, we report preferences expressed by the users regarding the layouts and using MR in public.