Human-in-the-loop anomaly detection in visualized data, where analysts visually inspect multivariate time series to spot changes in inter-variable relationships such as shifts in correlations or model fit, remains vital when domain expertise must complement automated analysis. Yet, the perceptual and cognitive mechanisms that enable or may bias this task are not well understood, limiting the design of intelligent visual interfaces. We conducted an online experiment with 212 participants who examined line-plot visualizations to detect anomalies arising from changes in the generative mechanism of a dependent variable. We modeled both detection accuracy and self-reported confidence using a combination of data-related metrics (information-theoretic complexity, similarity via correlation and mutual information, and exploratory psychophysical “JND-like” indicators) and user-related traits (cognitive style and subjective numeracy). Results show that chart complexity and similarity have a strong influence on anomaly detection, while individual differences further modulate both accuracy and confidence. These predictive models provide insight into the perceptual and cognitive processes underlying visual analysis, suggesting paths toward adaptive, user-aware interfaces that support decision-making regarding dynamic processes.

Decision support systems (DSSs) increasingly assist human users who should consider their output and combine it with independently available information when making a decision. Some decision-making events are dynamic, requiring a series of decisions in interdependent stages. In them, users may not use DSS advice optimally, possibly compromising task performance and preventing the DSS from fulfilling its purpose. We report an experiment in which participants made decisions in dynamic events, observing the output from a DSS. They largely followed normative decision-making rules, but decisions were based on the DSS's perceived, and not necessarily the actual, accuracy. Participants mostly made up their minds in the first stage, and using the DSS typically lengthened the time to reach a decision. These findings should be considered when designing DSS and related regulations and operational processes so people can utilize the DSS's full potential.

People typically enhance their online personas by sharing favourable personal information. Nevertheless, sharing of unfavourable information about oneself still occurs and is essential in some online contexts (e.g. allowing negative reviews). It remains unclear why people reveal potentially damaging information. We conducted an online experiment (N=462) to explore the effects of feedback properties and individual characteristics on online information sharing in two contexts (social and socioeconomic) where personal ratings are essential. We allowed users to conceal their personal rating if it dropped below a threshold. The context was the primary determinant of the threshold users chose. Control availability and feedback content triggered additional considerations and caused some users to change their (non)disclosure choices. However, many users relied on their priors (experience, assumptions) rather than on new information. Our findings show how people may fail to identify the impact of nondisclosure, which may signal undesirable information to others. These findings challenge the reliance on holding users solely accountable for their ‘informedness’ vis-à-vis disclosure of their personal information.

Intelligent systems support human operators’ decision-making processes, many of which are dynamic and involve temporal changes in the decision-related parameters. As we increasingly depend on automation, it becomes imperative to understand and quantify its influence on the operator’s decisions and to evaluate its implications for the human’s causal responsibility for outcomes. Past studies proposed a model for human responsibility in static decision-making processes involving intelligent systems. We present a model for dynamic, non-stationary decision-making events based on the concept of causation strength. We apply it to a test case of a dynamic binary categorization decision. The results show that for automation to influence humans significantly, it must have high detection sensitivity. However, this condition is insufficient since it is unlikely that automation, irrespective of its sensitivity, will sway humans with high detection sensitivity away from their original position. Specific combinations of automation and human detection sensitivities are required for automation to have a major influence. Moreover, the automation influence and the human causal responsibility that can be derived from it are sensitive to possible changes in the human’s detection capabilities due to fatigue or other factors, creating a “Responsibility Cliff.” This should be considered during system design and when policies and regulations are defined. This model constitutes a basis for further analyses of complex events in which human and automation sensitivity levels change over time and for evaluating human involvement in such events.

Information sharing on social networks is ubiquitous, intuitive, and occasionally accidental. However, people may be unaware of the potential negative consequences of disclosures, such as reputational damages. Yet, people use social networks to disclose information about themselves or others, advised only by their own experiences and the context-invariant informed consent mechanism. In two online experiments (N = 515 and N = 765), we investigated how to aid informed sharing decisions and associate them with the potential outcomes via notifications. Based on the measurements of sharing attitudes, our results showed that the effectiveness of informing the users via notifications may depend on the timing, content, and layout of the notifications, as well as on the users’ curiosity and rational cognitive style, motivating information processing. Furthermore, positive emotions may result in disregard of important information. We discuss the implications for user privacy and self-presentation. We provide recommendations on privacy-supporting system design and suggest directions for further research.

When humans interact with intelligent systems, their causal responsibility for outcomes becomes equivocal. We analyze the descriptive abilities of a newly developed responsibility quantification model (ResQu) to predict actual human responsibility and perceptions of responsibility in the interaction with intelligent systems. In two laboratory experiments, participants performed a classification task. They were aided by classification systems with different capabilities. We compared the predicted theoretical responsibility values to the actual measured responsibility participants took on and to their subjective rankings of responsibility. The model predictions were strongly correlated with both measured and subjective responsibility. Participants’ behavior with each system was influenced by the system and human capabilities, but also by the subjective perceptions of these capabilities and the perception of the participant's own contribution. A bias existed only when participants with poor classification capabilities relied less than optimally on a system that had superior classification capabilities and assumed higher-than-optimal responsibility. The study implies that when humans interact with advanced intelligent systems, with capabilities that greatly exceed their own, their comparative causal responsibility will be small, even if formally the human is assigned major roles. Simply putting a human into the loop does not ensure that the human will meaningfully contribute to the outcomes. The results demonstrate the descriptive value of the ResQu model to predict behavior and perceptions of responsibility by considering the characteristics of the human, the intelligent system, the environment, and some systematic behavioral biases. The ResQu model is a new quantitative method that can be used in system design and can guide policy and legal decisions regarding human responsibility in events involving intelligent systems.

Intelligent systems and advanced automation are involved in information collection and evaluation, decision-making, and the implementation of chosen actions. In such systems, human responsibility becomes equivocal. Understanding human causal responsibility is particularly important when systems can harm people, as with autonomous vehicles or, most notably, with autonomous weapon systems (AWSs). Using information theory, we developed a responsibility quantification (ResQu) model of human causal responsibility in intelligent systems and demonstrated its applications on decisions regarding AWS. The analysis reveals that comparative human responsibility for outcomes is often low, even when major functions are allocated to the human. Thus, broadly stated policies of keeping humans in the loop and having meaningful human control are misleading and cannot truly direct decisions on how to involve humans in advanced automation. The current model assumes stationarity, full knowledge regarding the characteristic of the human and automation, and ignores temporal aspects. It is an initial step toward the development of a comprehensive responsibility model that will make it possible to quantify human causal responsibility. The model can serve as an additional tool in the analysis of system design alternatives and policy decisions regarding human causal responsibility, providing a novel, quantitative perspective on these matters.

Users experience an almost continuous stream of information, making it necessary for them to decide to which information to attend and which to ignore. Indications about the relevance of specific information items can help. We report two experiments on the display of the relevance of information in the context of mobile devices, comparing different display formats and levels of resolution of the information. The results show that people prefer relatively few levels of relevance indications (two or three) and that numeric or graphic (bar) formats are usually preferred. We discuss some implications for the design of relevance indicators.

Users rarely ask for help, and they tend to respond negatively to unsolicited help, even when they can benefit from it. It is not clear whether this is due to the way systems provide help or whether people in general dislike unsolicited help. To address this issue the authors studied responses to solicited and unsolicited help from a human adviser regarding the use of an unfamiliar e-mail system. Novice users (the advisees) received spoken advice from distant experienced users (the advisers) upon the advisee’s request (the “pull” condition”) or the adviser’s initiative (the “push” condition”). Unsolicited advice helped performance more than advice requested by the advisee, but only for unfamiliar tasks and especially for difficult tasks. Although advisees perceived unsolicited advice as helpful, they were not interested in receiving such advice in the future. This study demonstrates that although advice can help to improve performance, it may still not be welcome, even when provided by a person.

The “intuitive” trust people feel when encountering robots in public spaces is a key determinant of their willingness to cooperate with these robots. We conducted four experiments to study this topic in the context of peacekeeping robots. Participants viewed scenarios, presented as static images or animations, involving a robot or a human guard performing an access-control task. The guards interacted more or less politely with younger and older male and female people. Our results show strong effects of the guard's politeness. The age and sex of the people interacting with the guard had no significant effect on participants' impressions of its attributes. There were no differences between responses to robot and human guards. This study advances the notion that politeness is a crucial determinant of people's perception of peacekeeping robots.

Preventive maintenance is essential for the smooth operation of any equipment. Still, people occasionally do not maintain their equipment adequately. Maintenance alert systems attempt to remind people to perform maintenance. However, most of these systems do not provide alerts at the optimal timing, and nor do they take into account the time required for maintenance or compute the optimal timing for a specific user. We model the problem of maintenance performance, assuming maintenance is time consuming. We solve the optimal policy for the user, i.e., the optimal timing for a user to perform maintenance. This optimal strategy depends on the value of user's time, and thus it may vary from user to user and may change over time. %We present a game Based on the solved optimal strategy we present a personalized maintenance agent, which, depending on the value of user's time, provides alerts to the user when she should perform maintenance. In an experiment using a spaceship computer game, we show that receiving alerts from the personalized alert agent significantly improves user performance.

The delivery of personalized news content depends on the ability to predict user interests. We evaluated different methods for acquiring user profiles based on declared and actual interest in various news topics and items. In an experiment, 36 students rated their interest in six news topics and in specific news items and read on 6 days standard, nonpersonalized editions and personalized (basic or adaptive) news editions. We measured subjective satisfaction with the editions and expressed preferences, along with objective measures, to infer actual interest in items. Users' declared interest in news topics did not strongly predict their actual interest in specific news items. Satisfaction with all news editions was high, but participants preferred the personalized editions. User interest was weakly correlated with reading duration, article length, and reading order. Different measures predicted interest in different news topics. Explicit measures predicted interest in relatively clearly defined topics such as sports, but were less appropriate for broader topics such as science and technology. Our results indicate that explicit and implicit methods should be combined to generate user profiles. We suggest that a personalized newspaper should contain both general information and personalized items, selected based on specific combinations of measures for each of the different news topics. Based on the findings, we present a general model to decide on the personalization of news content to generate personalized editions for readers.

There is limited knowledge on how to design effective interfaces to enable nonexpert users to interact with robot learning algorithms. This paper focuses on an interface design challenge: How to provide the user with sufficient information about the learned behavior. A simulated robotic task where the robot has online learning capabilities was developed. This platform was used to study the impact of the variability of the environmental conditions, the information provided on the relation between the learned robot behavior and the conditions in the environment, and the presence of a preview of the learned robot behavior on the use of the system and on task performance. The results show significant effects of the type and the quantity of displayed information. Forty-two participants made the best use of brief and contextualized notifications about changes in the environment: their presence improved the overall performance and the usage of the automation and reduced the workload. In contrast, adding previews of the learned behavior surprisingly impaired performance and reduced the use of automation.

We studied the effects of changes of system sensitivity over time and of providing notification about these changes on the responses to binary warnings in a simulated robotic task. Participants had to decide how much risk to take based on the output of a binary warning indicator. Experimental conditions differed in the presence or absence of notifications about changes of sensitivity of the warning system. Such changes happened twice during the experiment. The first change was an increase and the second a decrease of the sensitivity, returning it to its initial value. Participants successfully adapted their responses to the changes when notifications about the changes were displayed. However, without notifications, they did not properly adjust their response to changes, and their levels of compliance and reliance were differentially affected by the change, indicating a distinction between these two dimensions of trust.

Providing adaptive help during interaction with the system can be used to assist users in accomplishing their tasks. We propose providing guidance by highlighting the steps required for performing a task that the user intends to complete according to the prediction of a system. We present a study aimed at examining whether highlighting intended user steps in menus and toolbars as a means of assisting users in performing tasks is useful in terms of user response and performance. We also examined the effects of different accuracy levels of help and the control format on user response and performance. An experiment was conducted in which 64 participants performed tasks using menus and toolbars of a simulated email application. Participants were offered a highlighted guidance of the required steps in varying levels of accuracy (100%, 80%, 60% and no guidance). Our results support the benefits of highlighted help both in user performance times and in user satisfaction from receiving such assistance. Users found the assistance necessary and helpful and by the same token not unduly intrusive. Additionally, users felt that such assistance generally helped in reducing performance time on tasks. We did not find a significant difference when users receiving help at 80% accuracy was compared to those receiving help at 100% accuracy; however, such a difference does appear for those receiving 60% accuracy. In such cases, we found that the user's satisfaction level, perceived usefulness and trust in the system decreased while their notion of perceived intrusiveness increased. We conclude that assisting users by highlighting the required steps is useful so long as the minimal accuracy level of help is higher than 60%. Our study has implications on the implementation of highlighting next steps as a means of adaptive help and on integrating probability-based algorithms such as intention prediction to adaptive assistance systems.

Human interaction is highly intuitive: we infer reactions of our opponents mainly from what we have learned in years of experience and often assume that other people have the same knowledge about certain situations, abilities, and expectations as we do. In human-robot interaction (HRI) we cannot take for granted that this is equally true since HRI is asymmetrical. In other words, robots have different abilities, knowledge, and expectations than humans. They need to react appropriately to human expectations and behaviour. With this respect, scientific advances have been made to date for applications in entertainment and service robotics that largely depend on intuitive interaction. However, HRI today is often still unnatural, slow, and unsatisfactory for the human interlocutor. Both the sensorimotor interaction with environment and interlocutor, and the social aspects of the interaction still need to be researched and improved. Therefore, this full-day workshop aims to bring together researchers from different scientific fields to discuss these crosscutting issues and to exchange views on what are the preconditions and principles of intuitive interaction.

Mobile phones are rapidly becoming small-size general purpose computers, so-called smartphones. However, applications and data stored on mobile phones are less protected from unauthorized access than on most desktop and mobile computers. This paper presents a survey on users' security needs, awareness and concerns in the context of mobile phones. It also evaluates acceptance and perceived protection of existing and novel authentication methods. The responses from 465 participants reveal that users are interested in increased security and data protection. The current protection by using PIN (Personal Identification Number) is perceived as neither adequate nor convenient in all cases. The sensitivity of data stored on the devices varies depending on the data type and the context of use, asking for the need for another level of protection. According to these findings, a two-level security model for mobile phones is proposed. The model provides differential data and service protection by utilizing existing capabilities of a mobile phone for authenticating users.

Information systems increasingly provide options for visually inspecting data during the process of information discovery and exploration. Little research has dealt so far with user interactions with these systems, and specifically with the effects of characteristics of the displayed data and the user on performance with such systems. The study reports an experiment on users' performance with a visual exploration system. Participants had to identify target graphs within a large set of candidate graphs by using visual filtering criteria that differed in their efficiency for reducing the number of candidate graphs. A pay-off matrix and a time limit served to motivate users to select filter criteria efficiently. Performance was measured as the number of correct identifications of target graphs within a time-limit, and the number, type and position of filter criteria selected for the search. Efficiency was somewhat biased by users' preference to select filter criteria sequentially, starting from the left to the right. Rational and experiential cognitive styles affected performance, and they interacted with learning and the types of filter criteria chosen. The study shows that not only visual search tools can be used effectively but also that data and user characteristics affect task performance with such systems.

The editor's decision where and how to place items on a screen is crucial for the design of information displays, such as websites. We developed a statistical model that can facilitate automating this process by predicting the perceived importance of screen items from their location and size. The model was developed based on a 2-step experiment in which we asked participants to rate the importance of text articles that differed in size, screen location, and title size. Articles were either presented for 0.5 seconds or for unlimited time. In a stepwise regression analysis, the model's variables accounted for 65% of the variance in the importance ratings. In a validation study, the model predicted 85% of the variance of the mean apparent importance of screen items. The model also predicted individual raters' importance perception ratings. We discuss the implications of such a model in the context of automating layout generation. An automated system for layout generation can optimize data presentation to suit users' individual information and display preferences.

An ideal system should be usable and secure. However, increasing the security of a system often makes its use more cumbersome and less efficient. This tradeoff between usability and security poses major challenges for system designers. System security may be impaired when users override or ignore security features to facilitate the use of the system. Little empirical data are available on user behavior regarding the tradeoff between security and usability. To obtain such data we developed a controlled research environment (i.e., a microworld) for studying users’ tendency to take precautionary actions as a function of the tradeoff between a system’s usability and the level of security the system provides. It is a modified version of a “Tetris” game and includes an alert system that warns about possible virus attacks, which, if not prevented, can cause losses of monetary earnings. Users could alter the threshold settings of the security system. The system allows us to manipulate the usability cost of using a security feature, the severity of the consequences of an attack, the likelihood that a threat will occur, and the statistical properties of the security system. In a preliminary experiment two groups of 10 participants each used the system for three 20-minutes sessions. The likelihood for an attack was 4 times higher for one group than for the other group. The likelihood of an attack clearly affected participants' behavior. When attacks were more likely, participants altered thresholds more frequently, selected more cautious thresholds, and tended to respond more to security system alerts. This microworld is a step towards the development of quantitative predictive models of user interactions with security features while using a system.

Technological developments and the addition of new features to existing applications or services require the inclusion of security mechanisms to protect the user. When using these mechanisms the user faces a tradeoff between more risky and more efficient or safer and less efficient use of the system. We discuss this tradeoff and present a novel complementary experimental system which provides researchers and corporations the ability to explore and model the usability and security tradeoff in the context of user interaction with security systems and psychological acceptability, even before the actual development and implementation processes have ended.

The aim of this study was to examine how users perceive help on a mobile device with respect to the presentation format and the severity of the scenario the user encounters. We examined how 92 participants responded to four different formats of help offered for five different scenarios varying in their severity level using static images displayed in a web browser. The intrusiveness of the help format and the severity of the scenario affected participants' interest in receiving help. In certain conditions it may be helpful to suggest help, in particular when users do not know how to ask for it but show interest in receiving it when offered.

In two experiments participants had to detect changes in periodic sinusoidal functions, displayed in either graphic or tabular displays. Graphs had a major advantage over tables when the task required considering configurations of data. Both displays led to similar results when task performance could rely on inspecting individual data points. With graphs almost all participants reported using the optimal method for detecting changes in the function, i.e., they used the method requiring the least effort to perform the task. With tables only about half used the optimal detection method, and these participants showed transfer of learning of detection methods between tasks. Experience in using a detection method led to improved performance if the new task relied on the same method of detection. These findings demonstrate the need to consider task performance methods when determining the relative value of different displays. The set of tasks for which a display is used is likely to affect performance and needs to be analysed as a whole, since methods employed for one task can affect task performance in other tasks.

Abstract This paper describes ePaper, a research prototype system of a personalized newspaper on a mobile reading device. The ePaper aggregates content (i.e., news items) from various news providers, classifies the news items according to concepts from a news domain ontology, and delivers an electronic newspaper to each subscribed user (reader). The system personalizes the content of the newspaper according to the user's profiles and preferences by applying ontological content‐based and collaborative filtering algorithms. The user's profile is updated implicitly and dynamically, based on the tracking of their reading. Beyond personalization, the ePaper can also provide the user with a “standard edition” of a selected newspaper, as well as browsing capabilities in a repository of news items. The layout of the newspaper is adapted to the specifications of the reading device and to the user's preferences. In this overview paper, we highlight the main research challenges involved in the development of ePaper and describe how we addressed them.

Robotic systems in unstructured environments must cope with unknown, unpredictable, and dynamic situations. Inherent uncertainty, and limited sensor accuracy and reliability impede target recognition performance. Introducing a human operator into the system can help improve performance and simplify the robotic system. In this paper, four basic levels of collaboration were defined for human-robot collaboration in target recognition tasks. An objective function that includes operational and time costs was developed to quantify performance and determine the best collaboration level. Signal detection theory was applied to evaluate system performance. The optimal collaboration level for different cases was determined by using numerical analyses of the objective function. The findings indicate that the best system performance, the optimal values of performance measures, and the best collaboration level depend on the task, the environment, human and robot parameters, and the system characteristics. For the tested cases, the manual level was never the best collaboration level for achieving the optimal solution. The autonomous level was the best collaboration level when robot sensitivity was higher than human sensitivity. In general, collaboration of human and robot in target recognition tasks will improve upon the optimal performance of a single human detector.

Robots operating in unstructured environments must cope with uncertain, unpredictable and dynamic situations, resulting in lack of information. Introducing a human-operator into the system can help simplify the robotic system and improve performance. In previous work we presented a methodology to determine the best collaboration level for an integrated human-robot target recognition system in unstructured environments. The methodology is based on an objective function for target recognition in human-robot systems. In this paper the objective function was expanded to include operational and time costs that are both important for evaluating and optimizing system performance. Numerical analyses of the developed objective function, combined with signal detection theory, were applied for the predefined collaboration levels. Results indicate that optimal collaboration of human and robot in target recognition tasks will always improve the optimal performance of a single human detector.

This paper presents a methodology to determine the best collaboration level for an integrated human-robot target recognition system in unstructured environments. Four human-robot collaboration levels for target recognition tasks were defined, tested and evaluated. The collaboration levels were adjusted to an extensive range of automation, from manual to fully autonomous. An objective function for target recognition in human-robot systems was developed to allow computation of the expected value of system performance given the human, robot, environmental and task parameters. The objective function includes operational and time costs that are both important in evaluation and optimization of system performance. It quantifies the multitude of influencing parameters through a weighted sum of performance measures, and enables to predict system performance and the desirable level of collaborations and to help design optimal systems for specific tasks. Numerical analysis of the developed objective function combined with signal detection theory was applied for the defined collaboration levels. Based on the numerical analysis, the human and robot optimal values were determined for different task and environment parameters. Results indicate that collaboration of human and robot in target recognition tasks will always improve the optimal performance of a single human detector. When robot sensitivity is higher than human sensitivity, and, the overall system sensitivity never decreases beneath the robot sensitivity the best collaboration level is autonomous robot operation.

The assessment of the relative value of different design features for users is of great interest for software designers. Users' evaluations are generally measured through questionnaires. We suggest that other evaluation methods, including economic measures, may provide different estimates of the relative value of features. In a laboratory experiment we created four versions of a data-entry application by independently manipulating the system's usability and aesthetics. Users' evaluations of the four experimental systems were obtained in a within-subjects design. In addition, five between-subjects experimental conditions were created, based on the evaluation method (questionnaire alone or auction and questionnaire), monetary incentives (present or absent), and experience in using the system (present or absent). In questionnaire-based responses, the systems' usability affected evaluations of usability as well as aesthetics. Similarly, the systems' aesthetics affected evaluations of both aesthetics and usability. Questionnaire-based evaluations of usability and aesthetics were not affected by experience with the system or by monetary performance incentives. Auction bids were only influenced by the system's usability: bids corresponded to the objective performance levels that could be attained with the different systems. The results suggest that by using economic methods, researchers and practitioners can obtain system evaluations that are strongly related to performance criteria and that may be more valid when the evaluation context favors task-oriented performance.

Systems increasingly provide operators with the ability to move between different levels of automation from entirely manual modes, over partly automated modes (in which some task components are performed by the operator) and up to fully automated modes. It is still unclear what determines the relative efficiency of different levels of automation. An experimental system was developed to study these issues. In the system operators had to detect faulty items, based on partial information. Two levels of automation were tested: aided detection (operators received cues from a failure detection system), and approval (operators could change decisions of the failure detection system). An experiment assessed operators' performance with the different levels of automation and with low and high validity failure detection systems. Results showed that operators adjusted their response to the diagnostic value of the automation. Also, operators responded more strongly to low-validity cues in the approval condition than in the aided detection condition. These results point to some of the complex issues that need to be considered when choosing the appropriate level of automation for a system.

Two experiments assessed the effect of displaying a boundary on duration estimates and preference ratings for dynamic displays that were shown while users waited for the system's response. Displays were either symbolic (changing numbers) or graphic (increasing rectangles) and could contain a boundary that indicated when the interval was expected to be over. Duration estimates were similar for symbolic and graphic displays and for displays with and without a boundary. However, when the displays were encountered successively, participants assessed the graphic displays as having shorter durations than the symbolic displays. Faster rates of change in both types of displays led to increased duration estimates. Although displaying a boundary did not affect duration estimates, participants preferred displays in which a boundary was shown and preferred the graphic displays over the symbolic displays. Hence, bounded graphic displays are recommended as “wait” displays for computerized applications.