Augmented Reality is a technology that has been utilized in solutions to reduce the difficulties of human navigation. In this paper we survey a collection of these solutions, focusing on solutions with visualization elements meant to guide users. We propose a holistic framework for categorizing and understanding these solutions inspired by the Model-View-Controller software design pattern. The collected solutions are analyzed with our framework, and trends in the research are identified along with unsolved problems and potential future work.

Women are more likely to experience virtual reality (VR) sickness than men, which could pose a major challenge to the mass market success of VR. Because VR sickness often results from a visual-vestibular conflict, an effective strategy to mitigate conflict is to restrict the user’s field-of-view (FOV) during locomotion. Sex differences in spatial cognition have been well researched, with several studies reporting that men exhibit better spatial navigation performance in desktop three-dimensional environments than women. However, additional research suggests that this sex difference can be mitigated by providing a larger FOV as this increases the availability of landmarks, which women tend to rely on more than men. Though FOV restriction is already a widely used strategy for VR headsets to minimize VR sickness, it is currently not well understood if it impedes spatial learning in women due to decreased availability of landmarks. Our study (n=28, 14 men and 14 women) found that a dynamic FOV restrictor was equally effective in reducing VR sickness in both sexes, and no sex differences in VR sickness incidence were found. Our study did find a sex difference in spatial learning ability, but an FOV restrictor did not impede spatial learning in either sex.

Virtual reality sickness typically results from visual-vestibular conflict. Because self-motion from optical flow is driven most strongly by motion at the periphery of the retina, reducing the user’s field-of-view (FOV) during locomotion has proven to be an effective strategy to minimize visual vestibular conflict and VR sickness. Current FOV restrictor implementations reduce the user’s FOV by rendering a restrictor whose center is fixed at the center of the head mounted display (HMD), which is effective when the user’s eye gaze is aligned with head gaze. However, during eccentric eye gaze, users may look at the FOV restrictor itself, exposing them to peripheral optical flow which could lead to increased VR sickness. To address these limitations, we develop a foveated FOV restrictor and we explore the effect of dynamically moving the center of the FOV restrictor according to the user’s eye gaze position. We conducted a user study (n=22) where each participant uses a foveated FOV restrictor and a head-fixed FOV restrictor while navigating a virtual environment. We found no statistically significant difference in VR sickness measures or noticeability between both restrictors. However, there was a significant difference in eye gaze behavior, as measured by eye gaze dispersion, with the foveated FOV restrictor allowing participants to have a wider visual scan area compared to the head-fixed FOV restrictor, which confined their eye gaze to the center of the FOV.

Virtual reality (VR) has enjoyed significant popularity in recent years. Where navigation has been a fundamental appeal of 3D applications for decades, facilitating this in VR has been quite a challenge. Over the past decades, various virtual locomotion techniques (VLTs) have been developed that aim to offer natural, usable and efficient ways of navigating VR without inducing VR sickness. Several studies of these techniques have been conducted in order to evaluate their performance in various study conditions and virtual contexts. Taxonomies have also been proposed to either place similar techniques in meaningful categories or decompose them to their underlying design components. In this survey, we aim to aggregate and understand the current state of the art of VR locomotion research and discuss the design implications of VLTs in terms of strengths, weaknesses and applicability.

Recent studies show that women are more susceptible to visually-induced VR sickness, which might explain the low adoption rate of VR technology among women. Reducing field-of-view (FOV) during locomotion is already a widely used strategy to reduce VR sickness as it blocks peripheral optical flow perception and mitigates visual/vestibular conflict. Prior studies show that men are more adept at 3D spatial navigation than women, though this sex bias can be minimized by providing women with a larger FOV. Our study provides insight into the relationship between sex and FOV restriction with respect to VR sickness and spatial navigation performance which seem to conflict. We find the use of an FOV restrictor to be effective in mitigating VR sickness in both sexes while we did not find a negative effect of FOV restriction on spatial navigation performance.

Teleportation is a popular and low risk means of navigating in VR. Because teleportation discontinuously translates the user’s viewpoint, no optical flow is generated that could lead to vection-induced VR sickness. However, instant viewpoint translations and resulting discontinuous avatar representation is not only detrimental to presence and spatial awareness but also presents a challenge for gameplay design–particularly for multiplayer games. We compare out-of-body locomotion, a hybrid viewpoint technique that lets users seamlessly switch between a first-person and third-person avatar view, to traditional pointer-based teleportation. While in third-person, if the user doesn’t move, the camera remains stationary to avoid any optical flow generation. Third-person also lets users precisely and continuously navigate their avatar without risk of getting VR sick. The viewpoint automatically switches back to first-person as soon the users breaks line of sight with their avatar or the user requests to rejoin the avatar with a button press. A user study compares out-of-body locomotion to teleportation with participants (n=22) traversing an obstacle course. Results show that out-of-body locomotion requires significantly fewer (67%) viewpoint transitions than teleportation while there was no significant difference in performance. In addition to being able to offer a continuous avatar representation, participants also deemed out-of-body locomotion to be faster.

During terrestrial activities, sensation of pressure on the skin and tension in muscles and joints provides information about how the body is oriented relative to gravity and how the body is moving relative to the surrounding environment. In contrast, in aquatic environments when suspended in a state of neutral buoyancy, the weight of the body and limbs is offloaded, rendering these cues uninformative. It is not yet known how this altered sensory environment impacts virtual reality experiences. To investigate this question, we converted a full-face SCUBA mask into an underwater head-mounted display and developed software to simulate jetpack locomotion outside the International Space Station. Our goal was to emulate conditions experienced by astronauts during training at NASA's Neutral Buoyancy Lab. A user study was conducted to evaluate both sickness and presence when using virtual reality in this altered sensory environment. We observed an increase in nausea related symptoms underwater, but we cannot conclude that this is due to VR use. Other measures of sickness and presence underwater were comparable to measures taken above water. We conclude with suggestions for improved underwater VR systems and improved methods for evaluation of these systems based on our experience.

Teleportation is a popular locomotion technique that lets users safely navigate beyond the confines of available positional tracking space without inducing VR sickness. Because available walking space is limited and teleportation is faster than walking, a risk with using teleportation is that users might end up abandoning walking input and only relying on teleportation, which is considered detrimental to presence. We present redirected teleportation; an improved version of teleportation that uses iterative non-obtrusive reorientation and repositioning using a portal to redirect the user back to the center of the tracking space, where available walking space is larger. A user study compares the effectiveness, accuracy, and usability of redirected teleportation with regular teleportation using a navigation task in three different environments. Results show that redirected teleportation allows for a better utilization of available tracking space than regular teleportation, as it requires significantly fewer teleportations, while users walk more and use a larger portion of the available tracking space.

We demonstrate a novel method of applying the capabilities of mobile virtual reality technology to the health sciences by measuring physical exertion in a VR exercise game. By measuring changes in heart rate in a thirteen person user study, we find evidence to suggest that virtual reality exercise is able to induce a moderate to high level of physical exertion and produce an immersive an intriguing experience.

Navigating mobile virtual reality (VR) is a challenge due to limited input options and/or a requirement for handsfree interaction. Walking-in-place (WIP) is considered to offer a higher presence than controller input but only allows unidirectional navigation in the direction of the user's gaze--which impedes navigation efficiency. Leaning input enables omnidirectional navigation but currently relies on bulky controllers, which aren't feasible in mobile VR contexts. This note evaluates the use of head-tilt - implemented using inertial sensing - to allow for handsfree omnidirectional VR navigation on mobile VR platforms. A user study with 24 subjects compared three input methods using an obstacle avoidance navigation task: (1) head-tilt alone (TILT) (2) a hybrid method (WIP-TILT) that uses head tilting for direction and WIP to control speed; and (3) traditional controller input. TILT was significantly faster than WIP-TILT and joystick input, while WIP-TILT and TILT offered the highest presence. There was no difference in cybersickness between input methods.

Using real walking for virtual navigation generally delivers the most natural and immersive virtual reality experience, but its usage is generally bounded by available tracking space. To navigate beyond the confines of available tracking space, users need to switch to an artificial locomotion technique, such as controller input. However, having to switch from leg-based input to hand-based input is considered to break presence. We present a hybrid handsfree locomotion technique called legomotion that lets users seamlessly switch between real walking input and walking-in-place input to enable navigation at scale. A user study with 18 participants compared legomotion to full locomotion using a controller. Legomotion led to higher presence as switching to controller input was found to be more tedious. Because controller input is also faster than walking, we observed most users to abandon positional tracking input altogether and primary use a controller for navigation - which then led to a lower presence. This finding could have major implications for the design of VR locomotion.

People with visual impairments have a hard time getting consistent physical exercise, as they can not do some exercises, such as running outside, without a sighted guide. People with visual impairments have been shown to have higher spatial localization skills than sighted people, which lead us to believe that they could follow a drone on a running-track environment. This paper presents a feasibility study where we investigate the ability to localize and follow a low-cost flying drone in people with visual impairments. A Wizard of Oz style study was conducted with 2 blind participants. Our results indicate that blind individuals can accurately localize the drone and follow it. Qualitative results also indicate that the participants were comfortable with following the drone and had high efficacy when it came to following and localizing the drone. The study supports future development of a fully functioning prototype.

Low-cost smartphone adapters can bring virtual reality to the masses, but input is typically limited to using head tracking, which makes it difficult to perform complex tasks like navigation. Walking-in-place (WIP) offers a natural and immersive form of virtual locomotion that can reduce simulation sickness. WIP, however, is difficult to implement in mobile contexts as it typically relies on bulky controllers or an external camera. We present VR-STEP; a WIP implementation that uses real-time pedometry to implement virtual locomotion. VR-STEP requires no additional instrumentation outside of a smartphone's inertial sensors. A user study with 18 users compares VR-STEP with a commonly used auto-walk navigation method and finds no significant difference in performance or reliability, though VR-STEP was found to be more immersive and intuitive.

The accuracy of pedometry varies depending on where an inertial sensor is located on the body. Motivated by the increasing popularity of wearable computing, this paper investigates the accuracy with which pedometry can be achieved on a head-mounted device: something previous research has not investigated. A study with 16 subjects compares the accuracy of pedometry for walking and running with an inertial sensor located at the head, pocket and hand/arm. Our study did not detect a significant difference in step counting accuracy between sensor locations, which demonstrates the feasibility of pedometry-based apps for head-mounted displays.

Simulation of a visual impairment may lead to a better understanding of how individuals with visual impairments perceive the world around them and could be a useful design tool for interface designers to identify accessibility barriers. Current simulation tools, however, suffer from a number of limitations, pertaining cost, accuracy and immersion. We present a simulation tool (SIMVIZ) that mounts a wide angle camera on a head-mounted display to create a see-through stereoscopic display that simulates various types and levels of visual impairments. A qualitative user study evaluates the immersiveness, usability and effectiveness of SIMVIZ versus using a smartphone based simulator. SIMVIZ enables quick accessibility inspections during iterative software development.

Simulation of a visual impairment may lead to a better understanding of how individuals with visual impairments perceive the world around them and could be a useful design tool for interface designers to identify accessibility barriers. Current simulation tools, however, suffer from a number of limitations, pertaining cost, accuracy and immersion. We present a simulation tool (SimViz) that mounts a wide angle camera on a head-mounted display to create a see-through stereoscopic display that simulates various types and levels of visual impairments. SimViz enables quick accessibility inspections during iterative software development.

Traversing large open spaces is a challenging task for blind cane users, as such spaces are often devoid of tactile features that can be followed. Consequently, in such spaces cane users may veer from their intended paths. Wearable devices have great potential for assistive applications for users who are blind as they typically feature a camera and support hands and eye free interaction. We present HEADLOCK; a navigation aid for an optical head-mounted display that helps blind users traverse large open spaces by letting them lock onto a salient landmark across the space, such as a door, and then providing audio feedback to guide the user towards the landmark. A user study with 8 blind users evaluated the usability and effectiveness of two types of audio feedback (sonification and text-to-speech) for guiding a user across an open space to a doorway. Qualitative results are reported, which may inform the design of assistive wearable technology for users who are blind.

Traversing large open spaces is a challenging task for blind cane users, as such spaces are often devoid of tactile features that can be followed. Consequently, in such spaces cane users may veer from their intended paths. Wearable devices have great potential for assistive applications for users who are blind as they typically feature a camera and support hands and eye free interaction. We present HEADLOCK; a navigation aid for an optical head-mounted display that helps blind users traverse large open spaces by letting them lock onto a salient landmark across the space, such as a door, and then providing audio feedback to guide the user towards the landmark. HEADLOCK consists of interface modes for discovering landmarks, guiding a user towards a landmark, and recovering from an error state if a landmark is lost. HEADLOCK is designed with two forms of audio feedback: sonification and text-to-speech.

Loss of vision significantly impairs mobility, with blind individuals often relying on sighted individuals or public transportation to get around. Self-driving vehicles could significantly improve the mobility of blind people, but current legislation often requires a legal driver to be present in the vehicle who can take over in case of a malfunction. To enable blind people to eventually use a self-driving car independently, we present a steering interface that allows for steering a vehicle using haptic cues. User studies with six blind and sighted subjects identify what accuracy is required and possible using our interface to steer a vehicle on a track using a simulator. We investigate whether driving experience affects haptic steering performance and perform a qualitative study into the usability of our haptic steering interface.

Indoor localization and navigation systems for individuals with Visual Impairments (VIs) typically rely upon extensive augmentation of the physical space, significant computational resources, or heavy and expensive sensors; thus, few systems have been implemented on a large scale. This work describes a system able to guide people with VIs through indoor environments using inexpensive sensors, such as accelerometers and compasses, which are available in portable devices like smart phones. The method takes advantage of feedback from the human user, who confirms the presence of landmarks, something that users with VIs already do when navigating in a building. The system calculates the user's location in real time and uses it to provide audio instructions on how to reach the desired destination. Initial early experiments suggested that the accuracy of the localization depends on the type of directions and the availability of an appropriate transition model for the user. A critical parameter for the transition model is the user's step length. Consequently, this work also investigates different schemes for automatically computing the user's step length and reducing the dependence of the approach on the definition of an accurate transition model. In this way, the direction provision method is able to use the localization estimate and adapt to failed executions of paths by the users. Experiments are presented that evaluate the accuracy of the overall integrated system, which is executed online on a smart phone. Both people with VIs and blindfolded sighted people participated in the experiments, which included paths along multiple floors that required the use of stairs and elevators.

Glare significantly diminishes visual perception, and is a significant cause of traffic accidents. Existing haptic automotive interfaces typically indicate when and in which direction to steer, but they don't convey how much to steer, as a driver typically determines this using visual feedback. We present a novel haptic interface that relies on an intelligent vehicle position system to indicate when, in which direction and how far to steer, as to facilitate steering without any visual feedback. Our interface may improve driving safety when a driver is temporarily blinded, for example, due to glare or fog. Three user studies were performed, the first study tries to understand driving using visual feedback, the second study evaluates two different haptic encoding mechanisms with no visual feedback present, and a third study evaluates the supplemental effect of haptic feedback when used in conjunction with visual feedback. Studies show this interface to allow for blind steering in small curves and that it can improve a driver's lane keeping ability when combined with visual feedback.

Spatial perception is a challenging task for people who are blind due to the limited functionality and sensing range of hands. We present GIST, a wearable gestural interface that offers spatial perception functionality through the novel appropriation of the user's hands into versatile sensing rods. Using a wearable depth-sensing camera, GIST analyzes the visible physical space and allows blind users to access spatial information about this space using different hand gestures. By allowing blind users to directly explore the physical space using gestures, GIST allows for the closest mapping between augmented and physical reality, which facilitates spatial interaction. A user study with eight blind users evaluates GIST in its ability to help perform everyday tasks that rely on spatial perception, such as grabbing an object or interacting with a person. Results of our study may help develop new gesture based assistive applications.

In recent years, exercise games have been criticized for not being able to engage their players into levels of physical activity that are high enough to yield health benefits. A major challenge in the design of exergames, however, is that it is difficult to assess the amount of physical activity an exergame yields due to limitations of existing techniques to assess energy expenditure of exergaming activities. With recent advances in commercial depth sensing technology to accurately track players' motions in 3D, we present a technique called Vizical that uses a non-linear regression approach to accurately predict energy expenditure in real-time. Vizical may allow for creating exergames that can report energy expenditure while playing, and whose intensity can be adjusted in real-time to stimulate larger health benefits.

Whereas outdoor navigation systems typically rely upon GPS, indoor systems have to rely upon dierent techniques for localizing the user, as GPS signals cannot be received indoors. Over the past decade various indoor navigation systems have been developed. This paper provides a comprehensive overview of existing indoor navigation systems and analyzes the dierent techniques used for: (1) locating the user; (2) planning a path; (3) representing the environment; and (4) interacting with the user. Our survey identies a number of research issues that could facilitate large scale deployment of indoor navigation systems.

Indoor navigation systems for users who are visually impaired typically rely upon expensive physical augmentation of the environment or expensive sensing equipment; consequently few systems have been implemented. We present an indoor navigation system called Navatar that allows for localization and navigation by exploiting the physical characteristics of indoor environments, taking advantage of the unique sensing abilities of users with visual impairments, and minimalistic sensing achievable with low cost accelerometers available in smartphones. Particle filters are used to estimate the user's location based on the accelerometer data as well as the user confirming the presence of anticipated tactile landmarks along the provided path. Navatar has a high possibility of large-scale deployment, as it only requires an annotated virtual representation of an indoor environment. A user study with six blind users determines the accuracy of the approach, collects qualitative experiences and identifies areas for improvement.

Indoor localization and navigation systems for individuals with visual impairments (VI) typically rely upon extensive augmentation of the physical space or heavy, expensive sensors; thus, few systems have been adopted. This work describes a system able to guide people with VI through buildings using inexpensive sensors, such as accelerometers, which are available in portable devices like smart phones. The method takes advantage of feedback from the human user, who confirms the presence of landmarks. The system calculates the user's location in real time and uses it to provide audio instructions on how to reach the desired destination. Previous work suggested that the accuracy of the approach depended on the type of directions and the availability of an appropriate transition model for the user. A critical parameter for the transition model is the user's step length. The current work investigates different schemes for automatically computing the user's step length and reducing the dependency of the approach to the definition of an accurate transition model. Furthermore, the direction provision method is able to use the localization estimate and adapt to failed executions of paths by the users. Experiments are presented that evaluate the accuracy of the overall integrated system, which is executed online on a smart phone. Both people with visual impairments, as well as blindfolded sighted people, participated in the experiments. The experiments included paths along multiple floors, that required the use of stairs and elevators.

Proprioception --the human ability to sense the orientation of limbs without vision or hearing-- is one of the main drivers of motor operations and plays a significant role in input modalities such as touch and gestures. As an output modality proprioception has remained largely unexplored --though it can convey information to a user using their own body. Spatial interaction requires users to visually acquire the location of an object, which can then be manipulated using a touch or gesture. This is challenging if you are unable to see or in mobile contexts where the use of a display may be undesirable. This paper evaluates a tactile-proprioceptive display for eye and ear free 2D target acquisition and spatial interaction.

Many users with severe motor impairments, such as quadriplegics, interact with computers using an indirect selection technique called switch access scanning. Switch scanning allows for iteratively selecting an input from a set of input options using a single switch input and which replaces the use of a keyboard or a mouse, which they may be unable to use. Navigating an avatar in a 3D virtual world using existing switch access scanning systems is slow and erroneous because these interfaces are non-linear and requires players to provide continuous (holding a key) and mixed inputs (holding two or more keys). Through the analysis of navigation behavior of eight able-bodied users, a new scanning system called hold-and-release was developed. Using simulation hold-and-release scanning was found to be significantly more efficient than existing scanning systems. Multistep selection was found to be most efficient for mixing inputs, but expanding the selection set has no approximation errors.

Gesture-based interaction adds a new level of immersion to video games, but players who are blind are unable to play them as games use visual cues to indicate what gesture to provide and when. Though visual cues can be substituted with audio or haptic cues, this often requires access to the source code, which is not attainable for commercial games. We present a solution that uses real-time video analysis to detect the presence of a particular visual cue, which is then substituted with a vibrotactile cue that is provided with an external controller. A user study with 28 sighted participants with a popular commercial gesture based game showed no significant difference between visual and vibro-tactile cues, which demonstrates the feasibility of real-time sensory substitution as a cost-effective approach for making gesture-based video games accessible to players who are blind.

Visual cues are typically used in video games to indicate to the player what input to provide and when. Cues represented in multiple modalities that are presented simultaneously can be detected at lower thresholds, faster and more accurately than when presented separately in each modality. This characteristic has not been explored in playing video games to reduce errors. This paper explores the use of supplemental audio feedback to reduce errors in playing Bingo, a game which is typically played in crowded and noisy environments by a demographic, which -due to their age- are more likely to suffer from sensory impairments such as low vision or hearing impairments. A user study explored three different types of sonification (pitch, timbre, and audio icons) versus using no sonification and found that supplemental sonification using timbre or audio icons significantly reduces player's errors.

Lack of sight forms a significant barrier to participate in physical activity. Consequently, individuals with visual impairments are at greater risk for developing serious health problems, such as obesity. Exergames are video games that provide physical exercise. For individuals with visual impairments, exergames have the potential to reduce health disparities as they may be safer to play and can be played without the help of others. This paper presents VI Bowling, a tactile/audio exergame that can be played using an inexpensive motion-sensing controller. VI Bowling explores tactile dowsing: a novel technique for performing spatial sensorimotor challenges, which can be used for motor learning. VI Bowling was evaluated with six blind adults. All players enjoyed VI Bowling and the challenge tactile dowsing provided. Players could throw their ball with an average error of 9.76 degrees using tactile dowsing. Participants achieved an average active energy expenditure of 4.61 kJ/Min while playing VI Bowling, which is comparable to walking.

Lack of physical activity is a serious health concern for individuals who are visually impaired as they have fewer opportunities and incentives to engage in physical activities that provide the amounts and kinds of stimulation sufficient to maintain adequate fitness and to support a healthy standard of living. Exergames are video games that use physical activity as input and which have the potential to change sedentary lifestyles and associated health problems such as obesity. We identify that exergames have a number properties that could overcome the barriers to physical activity that individuals with visual impairments face. However, exergames rely upon being able to perceive visual cues that indicate to the player what input to provide. This paper presents VI Tennis, a modified version of a popular motion sensing exergame that explores the use of vibrotactile and audio cues. The effectiveness of providing multimodal (tactile/audio) versus unimodal (audio) cues was evaluated with a user study with 13 children who are blind. Children achieved moderate to vigorous levels of physical activity- the amount required to yield health benefits. No significant difference in active energy expenditure was found between both versions, though children scored significantly better with the tactile/audio version and also enjoyed playing this version more, which emphasizes the potential of tactile/audio feedback for engaging players for longer periods of time.

The immersive graphics, large amount of user-generated content, and social interaction opportunities offered by popular virtual worlds, such as Second Life, could eventually make for a more interactive and informative World Wide Web. Unfortunately, virtual worlds are currently not accessible to users who are visually impaired. This paper presents the work on developing TextSL, a client for Second life that can be accessed with a screen reader. Users interact with TextSL using a command-based interface, which allows for performing a plethora of different actions on large numbers of objects and avatars; characterizing features of such virtual worlds. User studies confirm that a command-based interface is a feasible approach towards making virtual worlds accessible, as it allows screen reader users to explore Second Life, communicate with other avatars, and interact with objects as well as sighted users. Command-based exploration and object interaction is significantly slower, but communication can be performed with the same efficiency as in the Second Life viewer. We further identify that at least 31% of the objects in Second Life lack a descriptive name, which is a significant barrier towards making virtual worlds accessible to users who are visually impaired.

Very few video games have been designed or adapted to allow people with vision impairment to play. Music/rhythm games however are particularly suitable for such people as they are perfectly capable of perceiving audio signals. Guitar Hero is a popular rhythm game yet it is not accessible to the visually impaired as it relies on visual stimuli. This paper explores replacing visual stimuli with haptic stimuli as a viable strategy to make games accessible. We developed a glove that transforms visual information into haptic feedback using small pager motors attached to the tip of each finger. This allows a blind player to play Guitar Hero. Several tests have been conducted and despite minor changes to the gameplay, visually impaired players are able to play the game successfully and enjoy the challenge the game provides. The results of the study also give valuable insights on how to make mainstream games blind-accessible.

The earliest design decisions often have a significant impact on software quality and are the most costly to revoke. One of the challenges in architecture design is to reduce the frequency of retrofit problems in software designs; not being able to improve the quality of a system cost effectively, a problem which frequently occurs during late stages. Software architecture assessment is essential in the design of a high quality system. However, assessing the effect of individual design decisions with respect to quality is often complicated by the fact that it is hard to identify exactly how particular qualities and quality factors are improved or impaired by design decisions. In this paper we present a framework that formalizes some of the relationships between software architecture and software quality; it compiles existing design knowledge (quality improving patterns) in a format suitable for architecture assessment. This framework may prevent the retrofit problem and can assist in reasoning about intra- and inter- quality tradeoffs. We illustrate our framework by creating an instance for it for the qualities usability, security and safety.