This paper bridges the gap in Human-Computer Interaction (HCI) research by comparatively assessing the effects of interpretability and outcome feedback on user trust and collaborative performance with AI. Through novel pre-registered experiments (N=1,511 total participants) using an interactive prediction task, we analyzed how interpretability and outcome feedback influence users’ task performance and trust in AI. The results counter the widespread belief that interpretability drives trust, showing that interpretability led to no robust improvements in trust and that outcome feedback had a significantly greater and more reliable effect. However, both factors had modest effects on participants’ task performance. These findings suggest that (1) interpretability may be less effective at increasing trust than factors like outcome feedback, and (2) augmenting human performance via AI systems may not be a simple matter of increasing trust in AI, as increased trust is not always associated with equally sizable performance improvements. Our exploratory analyses further delve into the mechanisms underlying this trust-performance paradox. These findings present an opportunity for research to focus not only on methods for generating interpretations but also on techniques that ensure interpretations impact trust and performance in practice.

Enterprises have recently adopted AI to human resource management (HRM) to evaluate employees’ work performance evaluation. However, in such an HRM context where multiple stakeholders are complexly intertwined with different incentives, it is problematic to design AI reflecting one stakeholder group's needs (e.g., enterprises, HR managers). Our research aims to investigate what tensions surrounding AI in HRM exist among stakeholders and explore design solutions to balance the tensions. By conducting stakeholder-centered participatory workshops with diverse stakeholders (including employees, employers/HR teams, and AI/business experts), we identified five major tensions: 1) divergent perspectives on fairness, 2) the accuracy of AI, 3) the transparency of the algorithm and its decision process, 4) the interpretability of algorithmic decisions, and 5) the trade-off between productivity and inhumanity. We present stakeholder-centered design ideas for solutions to mitigate these tensions and further discuss how to promote harmony among various stakeholders at the workplace.

Recently, Artificial Intelligence (AI) has been used to enable efficient decision-making in managerial and organizational contexts, ranging from employment to dismissal. However, to avoid employees’ antipathy toward AI, it is important to understand what aspects of AI employees like and/or dislike. In this paper, we aim to identify how employees perceive current human resource (HR) teams and future algorithmic management. Specifically, we explored what factors negatively influence employees’ perceptions of AI making work performance evaluations. Through in-depth interviews with 21 workers, we found that 1) employees feel six types of burdens (i.e., emotional, mental, bias, manipulation, privacy, and social) toward AI's introduction to human resource management (HRM), and that 2) these burdens could be mitigated by incorporating transparency, interpretability, and human intervention to algorithmic decision-making. Based on our findings, we present design efforts to alleviate employees’ burdens. To leverage AI for HRM in fair and trustworthy ways, we call for the HCI community to design human-AI collaboration systems with various HR stakeholders.

Instrumental variables (IVs) are a commonly used technique for causal inference from observational data. However, in the recent years, the use of the IV method has come under much criticism across multiple disciplines (e.g. [1] and [6] in Economics; [3] in Marketing; [5], [4], and [2] in Finance). This is because, in practice, the variation induced by IVs can be limited, which yields imprecise or biased estimates of causal effects and renders the approach ineffective for policy decisions. In this paper, we confront these challenges by formulating the problem of constructing instrumental variables from candidate exogenous information as a (supervised) machine learning problem that is amenable to the learning approach. We extend the standard learning framework to develop an algorithm we term MLIV (machine-learned instrumental variables), which allows training of instruments and causal inference to be simultaneously performed from sample data. We provide formal asymptotic theory and show O(√n) consistency and asymptotic normality for machine-learned instrument variables. We illustrate the effectiveness of the MLIVs in empirical environments consisting of both linear and nonlinear model parameters. Simulations and application to real-world data demonstrate that the algorithm is highly effective and significantly improves the performance of causal inference from observational data. The complete version of this paper can be found at https://ssrn.com/abstract=3352957.

We investigate the moderating effect of product attributes and consumer reviews on the efficacy of a collaborative filtering recommender system on an e-commerce site. We run a randomized field experiment on a top North American retailer's website with 184,375 users split into a recommender-treated group and a control group with 37,215 unique products in the dataset. By augmenting the dataset with Amazon Mechanical Turk tagged product attributes and consumer review data from the website, we study their moderating influence on recommenders in generating conversion. We first confirm that the use of recommenders increases the baseline conversion rate by 5.9%. We find that the recommenders act as substitutes for high average review ratings with the effect of using recommenders increasing the conversion rate as much as about 1.4 additional average star ratings. Additionally, we find that the positive impacts on conversion from recommenders are greater for hedonic products compared to utilitarian products while search-experience quality did not have any impact. We also find that the higher the price, the lower the positive impact of recommenders, while having lengthier product descriptions and higher review volumes increased the recommender's effectiveness. More findings are discussed in the Results.

With the growing popularity of search engines among consumers, advertising on search engines has also grown considerably. We study optimal bidding strategies for advertisers in sponsored search auctions. In general, these auctions are run as variants of second-price auctions but have been shown to be incentive incompatible. Thus, advertisers have to be strategic about bidding. Uncertainty in the decision-making environment, budget constraints and the presence of a large portfolio of keywords makes the bid optimization problem non-trivial. In addition, there keywords are not independent. In this paper, we formulate and solve the advertiser's decision problem. We propose two bidding policies in our paper. The first policy ignores the interaction between keywords and is referred to as the "myopic" policy in this paper. We extend this bidding policy to incorporate interaction between keywords, and refer to this policy as the "forward-looking" policy since it entails decision making over several time horizons. Depending on the advertiser's intent, level of sophistication and nature of the products being advertised, the advertiser might choose the myopic or the forward-looking policy. This paper makes three main contributions. The first contribution is towards improving managerial practice. Advertisers spend billions of dollars on sponsored search. The techniques described in the paper can help increase the Return on Investment (RoI) for advertisers and SEM firms, as demonstrated in our field implementation. The second key contribution is that our approach represents a significant step forward for the academic literature on bidding in multi-slot auctions. All the papers to date have studied the problem either in a deterministic setting or in a single-slot setting and have relied on heuristic solution techniques due to the complexity of the optimization problem. In contrast, we compute optimal bids in the more realistic stochastic multi-slot setting. The third contribution of this paper is that it is the first paper on bidding in sponsored search to incorporate the interdependence between keywords into a multi-period bidding problem. The interdependence in keyword performance, commonly referred to as spillovers, is a well-documented feature of sponsored search (Rutz and Bucklin, 2011) but has not been considered in the bidding literature. The interactions between keywords are modeled in the form of positive spillovers from generic keywords into branded keywords. The spillovers are estimated using a dynamic linear model framework. An approximate dynamic program is formulated to solve the advertiser's bidding problem. To validate our approaches, we estimate the parameters of the models using data from an advertiser's sponsored search campaign and use the bids proposed by the models in a field experiment. The "myopic" policy outperforms the advertiser's policy by 75.38% and the "forward looking" policy improves the ROI by another 7.87%. The results of the field implementation show that the proposed bidding techniques are very effective in practice.

There has been significant recent interest in studying consumer behavior in sponsored search advertising (SSA). Researchers have typically used daily data from search engines containing measures such as average bid, average ad position, total impressions, clicks and cost for each keyword in the advertiser's campaign. A variety of random utility models have been estimated using such data and the results have helped researchers explore the factors that drive consumer click and conversion propensities. However, virtually every analysis of this kind has ignored the intra-day variation in ad position. We show that estimating random utility models on aggregated (daily) data without accounting for this variation will lead to systematically biased estimates -- specifically, the impact of ad position on click-through rate (CTR) is attenuated and the predicted CTR is higher than the actual CTR. First, we prove that the average daily position of an ad is less in convex order than the actual position of the ad for an impression. Using this result, we analytically demonstrate the existence of the aggregation bias. Second, using a large disaggregate dataset from a major search engine containing 8 million impressions, we empirically validate our findings for both the traditional logit model and the Hierarchical Bayesian models that are commonly used in the SSA literature. Third, we build a game-theoretic model to analyze the effect of the bias on the equilibrium of the SSA auction.We find that advertisers bid lower in SSA auctions as a result of the bias, which always leads to lower search-engine revenue. We also find that an advertiser can always increase his payoff when he unilaterally switches to complete data from aggregate data. Finally, we empirically quantify the losses experienced by the search engine and the advertisers and find that the search engine loses over 17% of its revenue on average. We also observe that an advertiser loses around 6% of his payoffs due to data aggregation. Our findings raise serious concerns for SSA practitioners and also question the adequacy of the data standards that have become common in SSA. Finally, we provide recommendations for aggregate datasets that do not suffer from the bias.

This document presents the extended abstract for "Recommender Systems and Their Effect on Consumers" and a link to the full paper on SSRN.

New network technologies constantly seek to displace incumbents. Their success depends on technological superiority, the size of the incumbent's installed base, users' adoption behaviors, and various other factors. The goal of this paper is to develop an understanding of competition between network technologies and identify the extent to which different factors, in particular converters (a.k.a. gateways), affect the outcome. Converters can help entrants overcome the influence of the incumbent's installed base by enabling cross-technology interoperability. However, they have development, deployment, and operations costs and can introduce performance degradations and functionality limitations, so that if, when, why, and how they help is often unclear. To this end, the paper proposes and solves a model for adoption of competing network technologies by individual users. The model incorporates a simple utility function that captures key aspects of users' adoption decisions. Its solution reveals a number of interesting and at times unexpected behaviors, including the possibility for converters to reduce overall market penetration of the technologies and to prevent convergence to a stable state, something that never arises in their absence. The findings were tested for robustness, e.g., different utility functions and adoption models, and found to remain valid across a broad range of scenarios.

We study optimal bidding strategies for advertisers in budget constrained multi-item multi-slot auctions. The classic application context is that of bidding in sponsored search auctions. In general, these auctions are run as variants of second-price auctions but have been shown to be incentive incompatible. Thus advertisers have to be strategic about bidding. Uncertainty in the decision-making environment, budget constraints and the presence of a large portfolio of candidate keywords makes the optimization problem non-trivial. We first present an analytical model to characterize the optimal bidding strategy and illustrate its application using real-world data. We find that the proposed strategies are very effective in practice.

An important problem in search engine advertising is key-word1 generation. In the past, advertisers have preferred to bid for keywords that tend to have high search volumes and hence are more expensive. An alternate strategy involves bidding for several related but low volume, inexpensive terms that generate the same amount of traffic cumulatively but are much cheaper. This paper seeks to establish a mathematical formulation of this problem and suggests a method for generation of several terms from a seed keyword. This approach uses a web based kernel function to establish semantic similarity between terms. The similarity graph is then traversed to generate keywords that are related but cheaper.

This paper examines the effect of recommender systems on the diversity of sales. Two anecdotal views exist about such effects. Some believe recommenders help consumers discover new products and thus increase sales diversity. Others believe recommenders only reinforce the popularity of already popular products. This paper is a first attempt to reconcile these seemingly incompatible views. We explore the question in two ways. First, modeling recommender systems analytically allows us to explore their path dependent effects. Second, turning to simulation, we increase the realism of our results by combining choice models with actual implementations of recommender systems. We arrive at three main results. One, some common recommenders lead to a net reduction in average sales diversity. Two, there exists path dependence, and in individual instances the same recommender can either increase or decrease diversity. Three, we show how basic design choices affect the outcome.

No abstract available.

Distributed IR systems query a large number of IR servers, merge the retrieved results and display them to users. Since different servers handle collections of different sizes, have different processing and bandwidth capacities, there can be considerable heterogeneity in their response times. The broker in the distributed IR system thus has to make decisions regarding terminating searches based on perceived value of waiting -- retrieving more documents -- and the costs imposed on users by waiting for more responses. In this paper, we apply utility theory to formulate the broker's decision problem. The problem is a stochastic nonlinear program. We use Monte Carlo simulations to demonstrate how the optimal wait time may be determined in the context of a comparison shopping engine that queries multiple store websites for price and product information. We use data gathered from 30 stores for a set of 60 books. Our research demonstrates how a broker can leverage information about past retrievals regarding distributions of server response time and relevance scores to optimize its performance. Our main contribution is the formulation of the decision model for optimal wait time and proposal of a solution method. Our results suggest that the optimal wait time is highly sensitive to the manner in which users value from a set of retrieved results differs from the sum of user value from each result evaluated independently. We also find that the optimal wait time increases with the size of the distributed collections, but only if user utility from a set of results is nearly equal to the sum of utilities from each result.

Summary form only given. Emerging caching techniques such as differential caching, push caching, etc enable preferential treatment of premium objects in a cache. By the use of appropriate placement, replacement or reporting policies, a service provider can offer multiple levels of QoS for different objects. For example, certain objects may be assigned higher priorities and be placed in a premium cache before they get pushed into the regular cache to increase their lifetime in the cache. Similar techniques open avenues for a service provider such as an IAP (Internet Access Provider) to offer several quality-differentiated services to content publishers at different prices and thus maximize revenues. We focus on the optimal pricing strategy of a caching service provider offering multiple quality levels. We analyze trade-offs between bandwidth savings and the potential to charge publishers for caching their content. We determine the optimum number of quality levels to offer and motivate a means for determining optimal allocation of the cache space among the different service classes.