The ever-increasing number of data breaches targeting user credentials has become undeniable reality in our digital age. Although there are third-party breach notification services (e.g., Have I Been Pwned, Firefox Monitor) that allow users to check whether their credentials have been involved in data breaches, the number of users who are aware of or use such services may be limited. To better inform users about the breach status of their accounts, we designed a prototype registration system where the website uses readily available information (e.g., email address) to check whether this account was involved in a data breach, and then inform the user about the breach status of the email address while signing up for a website. We investigated the effectiveness of this system by performing an online study (n = 373) in which participants were asked to register to a mock-up website that presented them with a data breach notification based on Protection Motivation Theory (PMT). We found that 64% of participants were exposed in one or more breaches. A follow-up survey, 3 days after the initial survey, was conducted to determine if there were any behavior changes (e.g., changing passwords) of participants whose accounts were involved in some data breaches. We found that 40% of the participants changed their passwords on their some accounts. Finally, we present our qualitative data analysis to shed light on participants’ motivations behind their decisions as well as their perceptions of the integration of our prototype registration system.

In the context of teamwork, prior efforts noted that trust degradation due to failures is slower in the case of human partners compared to automated agents. In our work, we wanted to investigate whether this holds true when the cost of mistakes (i.e., risk) is high. Toward that, we designed a 2 (partner: automation/human) × 2 (risk: low/high) × 2 (reliability: low/high) between-group study where participants completed five rounds of a target identification task with a partner (either automation or human). The findings suggest that users’ perceived trustworthiness of the partner is affected by reliability groups appropriately, irrespective of risk groups. However, with human partners, although participants fail to calibrate dependence appropriately in low-risk groups, dependence is appropriate in high-risk groups. The finding suggests that the effect of human-like characteristics on trust should not be considered independent of risk. The implications of our findings are discussed in the paper.

Safety-critical systems are often equipped with warning mechanisms to alert users regarding imminent system failures. However, they can suffer from false alarms, and affect users’ emotions and trust in the system negatively. While providing feedback could be an effective way to calibrate trust under such scenarios, the effects of feedback and warning reliability on users’ emotions, trust, and compliance behavior are not clear. This article investigates this by designing a 2 (feedback: present/absent) × 2 (warning reliability: high/low) × 4 (sessions) mixed design study where participants interacted with a simulated unmanned aerial vehicle (UAV) system to identify and neutralize enemy targets. Results indicated that feedback containing both correctness and affective components decreased users’ positive emotions and trust in the system, and increased loneliness and hostility (negative) emotions. Emotions were found to mediate the relationship between feedback and trust. Implications of our findings for designing feedback and calibration of trust are discussed in the paper.

Prior efforts have noted the effect of reliability, risk, and degree of anthropomorphism on trust in the context of human-agent interaction. However, the effects of these factors on resulting emotions while interacting with autonomous agents and their influence on trust are not clear. Towards that, we designed a 2 (partner: automation/human) × 2 (risk: low/high) × 2 (reliability: low/high) between-group study to identify relevant discrete emotions and their (emotions’) influences on users’ trustworthiness perceptions (ability, integrity, and benevolence). The results identified four emotion factors (positive emotions, hostility, anxiety, and loneliness) related to human-agent interaction. Although the reliability condition affected all four emotion factors, the mediating effects of the emotion factors on reliability and trustworthiness perceptions relationships differed for the varying emotion factors. The implications of our findings for trust calibration in the context of designing interactive systems are discussed in the paper.

Anthropomorphism in the design of interface agents is implicitly linked to increasing user trust and acceptance. However, the role of perceived anthropomorphism and perceived trustworthiness in trust appropriateness given a system’s capabilities and limitations is unclear. We designed a 2 (reliability: low, high) x 3 (agent appearance: computer, avatar, human) between-subject study to observe how agent appearance influenced user perceptions of and reliance on an automated teammate in a collaborative image classification task. Trust appropriateness was characterized as the degree to which reliance matched an optimal level given the system’s reliability. Although agent appearance did not significantly influence trust appropriateness, it did affect perceptions of trustworthiness, particularly for low reliability agents. Our results suggest that trust and anthropomorphism involve highly related, dynamic perceptions aimed at anticipating system behavior. Based on our findings, recommendations for future research on trust and anthropomorphism are discussed along with some design implications.

This study focuses on identifying the factors contributing to a sense of personal responsibility that could improve understanding of insecure cybersecurity behavior and guide research toward more effective messaging targeting non-adopting populations. Towards that, we ran a 2(account type) x2(usage scenario) x2(message type) between-group study with 237 United States adult participants on Amazon MTurk, and investigated how the non-adopting population allocates blame, and under what circumstances they blame the end user among the parties who hold responsibility: the software companies holding data, the attackers exposing data, and others. We find users primarily hold service providers accountable for breaches but they feel the same companies should not enforce stronger security policies on users. Results indicate that people do hold end users accountable for their behavior in the event of a breach, especially when the users' behavior affects others. Implications of our findings in risk communication is discussed in the paper.

Setting expectations for future behavior with promises is one way to manage human-human trusting relationships. To investigate the effect of promises made by an automated system, we conducted a 2 (reliability: low, high) x 3 (promise type: no-promise, optimistic, realistic) between-subject study where participants collaborated with an Automated Target Detection (ATD) system to classify images in multiple rounds of gameplay. We found that an optimistic promise (i.e., "I promise to do better") initially led to significantly more reliance on automation than a realistic promise (i.e., "I cannot do better than this"), but not in the long-term. High reliability participants relied more on the automation and reported greater perceived trustworthiness compared to low reliability participants. In addition, participants in the no promise group reported a greater degree of frustration compared to the other groups. We discuss the implications of our findings for trust repair in automated systems.

Trust in automation has been largely studied through a cognitive lens, though theories suggest that emotions play an important role. Understanding the affective aspects of human-automation trust can inform the design of systems that garner appropriate trust calibration. Toward this, we designed 4 videos describing a hypothetical drone system: one control, and three with additional performance or process information, or both. Participants reported the intensity of 19 emotions they would anticipate as system operator, perceptions of the system’s trustworthiness, individual differences, and perceptions of the institution behind the system. Emotions factored into hostility, positive, anxiety, and loneliness components that were regressed on system information, individual differences, and institutional trust. We found that financial risk-taking, recreational risk-taking, and propensity to trust influenced the intensity of different emotion factors. Moreover, greater perceptions of the institution’s ability led to more intense hostility emotions, greater perceptions of the institution’s benevolence led to less intense hostility, and integrity perceptions decreased anxiety and increased positive emotions. Lastly, structural assurance led to less intense hostility and anxiety and more intense positive emotions. These results offer support for the relationship between human-automation trust and emotions, warranting future research on how operator emotions can be addressed to improve trust calibration.

To promote safe and effective human-computer interactions, researchers have begun studying mechanisms for "trust repair" in response to automated system errors. The extent to which users distinguish between a system and the system's developers may be an important factor in the efficacy of trust repair messages. To investigate this, we conducted a 2 (reliability) x 3 (blame) between-group, factorial study. Participants interacted with a high or low reliability automated system that attributed blame for errors internally ("I was not able..."), pseudo-externally ("The developers were not able..."), or externally ("A third-party algorithm that I used was not able..."). We found that pseudo-external blame and internal blame influenced subjective trust differently, suggesting that the system and its developers represent distinct trustees. We discuss the implications of our findings for the design and study of human-automation trust repair.

Safety-critical systems (e.g., UAV systems) often incorporate warning modules that alert users regarding imminent hazards (e.g., system failures). However, these warning systems are often not perfect, and trigger false alarms, which can lead to negative emotions and affect subsequent system usage. Although various feedback mechanisms have been studied in the past to counter the possible negative effects of system errors, the effect of such feedback mechanisms and system errors on users’ immediate emotions and task performance is not clear. To investigate the influence of affective feedback on participants’ immediate emotions, we designed a 2 (warning reliability: high/low) × 2 (feedback: present/absent) between-group study where participants interacted with a simulated UAV system to identify and neutralize enemy vehicles under time constraint. Task performance along with participants’ facial expressions were analyzed. Results indicated that giving feedback decreased fear emotions during the task whereas warning increased frustration for high reliability groups compared to low reliability groups. Finally, feedback was found not to affect task performance.

In safety-critical systems, it is essential to communicate relevant information to facilitate decision-making, promote trust, and improve performance without overloading users. To explore the effect of system performance information on rational and emotional processing by users, we performed a between-subject experiment in which participants were asked to imagine themselves as a drone operator or system administrator in a high-, medium-, or low-risk scenario. Then, based on their imagined scenario and role, participants rated the relevance of four aspects of system reliability to decision-making with the system, as well as the expected intensity of the GREAT emotions. Results indicate that system performance information affected participants’ reasoning differently depending on risk level. Moreover, participants had different perspectives depending on their role in the system. Those in administrator roles indicated higher respect ratings for those with a similar role. These findings demonstrate that contextual risk and a user’s role can influence emotions and attitudes toward safety-critical computer systems.

Prior work notes dispositional, learned, and situational aspects of trust in automation. However, no work has investigated the relative role of these factors in initial trust of an automated system. Moreover, trust in automation researchers often consider trust unidimensionally, whereas ability, integrity, and benevolence perceptions (i.e., trusting beliefs) may provide a more thorough understanding of trust dynamics. To investigate this, we recruited 163 participants on Amazon's Mechanical Turk (MTurk) and randomly assigned each to one of 4 videos describing a hypothetical drone system: one control, the others with additional system performance or process, or both types of information. Participants reported on trusting beliefs in the system, propensity to trust other people, risk-taking tendencies, and trust in the government law enforcement agency behind the system. We found that financial risk-taking tendencies influenced trusting beliefs. Also, those who received process information were likely to have higher integrity and ability beliefs than those not receiving process information, while those who received performance information were likely to have higher ability beliefs. Lastly, perceptions of structural assurance positively influenced all three trusting beliefs. Our findings suggest that a) users' risk-taking tendencies influence trustworthiness perceptions of systems, b) different types of information about a system have varied effects on the trustworthiness dimensions, and c) institutions play an important role in users' calibration of trust. Insights gained from this study can help design training materials and interfaces that improve user trust calibration in automated systems.

Apache Spark provides numerous configuration settings that can be tuned to improve the performance of specific applications running on the platform. However, due to its multi-stage execution model and high interactive complexity across nodes, it is nontrivial to understand how/why a specific setting influences the execution flow and performance. To address this challenge, we develop an execution model-driven framework that extracts key performance metrics relevant to different levels of execution (e.g., application level, stage level, task level, system level) and applies statistical analysis techniques to identify the key execution features that change significantly in response to changes in configuration settings. This allows users to answer questions such as "How does configuration setting X affect the execution behavior of Spark?" or "Why does changing configuration setting X degrade the performance of Spark application Y?". We tested our framework using 6 open source applications (e.g., Word Count, Tera Sort, KMeans, Matrix Factorization, PageRank, and Triangle Count) and demonstrated the effectiveness of our framework in identifying the underlying reasons behind changes in performance.

In large-scale data stream management systems, sampling rate of different sensors can change quickly in response to changed execution environment. However, such changes can cause significant load imbalance on the back-end servers, leading towards performance degradation and data loss. To address this challenge, in this paper, we present a model-driven middleware service (i.e., Arion) that uses a two-step approach to minimize data loss. Specifically, Arion constructs models and algorithms for overload prediction for heterogeneous systems (where different streams can have different sampling rates and message sizes) leveraging limited execution traces from homogeneous systems (where each stream has the same sampling rate and message size). Subsequently, when an overload condition is predicted (or detected), Arion first leverages the a priori constructed models to identify the streams (if any) that can be split into multiple substreams to scale up the performance and minimize data loss without allocating additional servers. If the software based solution turns out to be inadequate, in the second stage, the system allocates additional servers and redirects streams to stabilize the system leveraging the models. Extensive evaluation on a 6 node cluster using Apache Cassandra for various scenarios shows that our approach can predict the potential overload condition with high accuracy (81.9%) while minimizing data loss and the number of additional servers significantly.

This article investigates the effectiveness of informational videos that are designed to provide an introduction to two-step verification (i.e., 2FA) and in turn seeks to improve the adoption rate of 2FA among users. Toward that, eight video tutorials based on three themes (e.g., Risk, Self-efficacy, and Contingency) were designed, and a three-way between-group study with 399 participants on Amazon’s MTurk was conducted. Furthermore, a follow-up study was run to see the changes in participants’ behavior (e.g., enabling of 2FA). The Self-efficacy and Risk themes were found to be the most effective in making the videos more interesting, informative, and useful. Willingness to try 2FA was found to be higher for participants who were exposed to both the Risk and Self-efficacy themes. Participants’ decisions regarding actually enabling 2FA was found to be significantly correlated with how interesting, informative, and useful the videos were. Implications of our findings in a broader context are discussed in the article.

This paper investigates a location-based authentication system where authentication questions are generated based on users' locations tracked by smartphones. More specifically, the system builds a location profile for a user based on periodically logged Wi-Fi access point beacons over time, and leverages this location profile to generate authentication questions. To evaluate the various aspects of this location-based authentication approach, we deployed the application on users' smartphones and conducted a real-life study for one month with 14 users. To simulate various kinds of adversaries (e.g., Naive vs. Knowledgeable), in our study, we recruited volunteers in pairs (e.g., Friends), in addition to single participants. Over the course of the experiment, each user is periodically presented with two sets of authentication questions. The first set is generated based on a user's own data. The second set is generated based on a randomly selected user's data. Additionally, in cases of paired participants, each user is presented with a third set of questions which is generated based on the user's friend's data. In each case, three different kinds of questions of varying difficulty levels are generated and presented to the user. Finally, we present a Bayesian classifier based authentication algorithm that can authenticate legitimate users with high accuracy by leveraging individual response patterns. We also discuss various aspects of location-based authentication mechanisms based on our findings in this paper.

Integration of NFC radios into smartphones is expediting the adoption of mobile devices as the preferred method for accessing physical locations, bank accounts, and other valuable resources. The pervasive nature of authentication using these mobile devices, however, comes with increased security considerations stemming from the possibility of physical loss of the device. To minimize the risk caused by stolen devices, this paper introduces a method for confirming the identity of a device's user based on her recent macroscopic behavior over space and time. The user's behavior is continuously recorded by a set of devices embedded in the environment (e.g., Wi-Fi Access Point) and used to train a probabilistic n-gram model. Subsequently, deviations caused by stolen devices can be detected by comparing the user's recent behavior against the trained model. Our first evaluation results demonstrate the ability of the proposed approach to detect anomalies in the user's behavior without generating a significant number of false alarms.

Integration of Near Field Communication (NFC) sensors into mobile devices has enabled their use for authentication. The ubiquitous nature of authentication using mobile devices comes though with increased security considerations. In addition to the risk of being stolen, mobile devices are increasingly susceptible to different types of software attacks. User-provided passwords such as Personal Identification Numbers (PINs) are often employed to ameliorate these limitations. The use of passwords though, has its own vulnerabilities that are mainly caused by the passwords' static nature and low entropy. To eliminate the security threats caused by untrusted devices and static passwords, we propose the development of new types of biometric authentication, based on macroscopic human behavior.