Learning of preference models from human feedback has been central to recent advances in artificial intelligence. Motivated by the cost of obtaining high-quality human annotations, we study efficient human preference elicitation for learning preference models. The key idea in our work is to generalize optimal designs, an approach to computing optimal information-gathering policies, to lists of items that represent potential questions with answers. The policy is a distribution over the lists and we elicit preferences from them proportionally to their probabilities. To show the generality of our ideas, we study both absolute and ranking feedback models on items in the list. We design efficient algorithms for both and analyze them. Finally, we demonstrate that our algorithms are practical by evaluating them on existing question-answering problems.

We study the problem of best-arm identification (BAI) in the fixed-budget setting with heterogeneous reward variances. We propose two variance-adaptive BAI algorithms for this setting: SHVar for known reward variances and SHAdaVar for unknown reward variances. Our algorithms rely on non-uniform budget allocations among the arms where the arms with higher reward variances are pulled more often than those with lower variances. The main algorithmic novelty is in the design of SHAdaVar, which allocates budget greedily based on overestimating the unknown reward variances. We bound probabilities of misidentifying the best arms in both SHVar and SHAdaVar. Our analyses rely on novel lower bounds on the number of pulls of an arm that do not require closed-form solutions to the budget allocation problem. Since one of our budget allocation problems is analogous to the optimal experiment design with unknown variances, we believe that our results are of a broad interest. Our experiments validate our theory, and show that SHVar and SHAdaVar outperform algorithms from prior works with analytical guarantees.

When humans collaborate with each other, they often make decisions by observing others and considering the consequences that their actions may have on the entire team, instead of greedily doing what is best for just themselves. We would like our AI agents to effectively collaborate in a similar way by capturing a model of their partners. In this work, we propose and analyze a decentralized Multi-Armed Bandit (MAB) problem with coupled rewards as an abstraction of more general multi-agent collaboration. We demonstrate that naive extensions of single-agent optimal MAB algorithms fail when applied for decentralized bandit teams. Instead, we propose a Partner-Aware strategy for joint sequential decision-making that extends the well-known single-agent Upper Confidence Bound algorithm. We analytically show that our proposed strategy achieves logarithmic regret, and provide extensive experiments involving human-AI and human-robot collaboration to validate our theoretical findings. Our results show that the proposed partner-aware strategy outperforms other known methods, and our human subject studies suggest humans prefer to collaborate with AI agents implementing our partner-aware strategy.

This paper proposes a novel pipeline for automatic grammar augmentation that provides a significant improvement in the voice command recognition accuracy for systems with small footprint acoustic model (AM). The improvement is achieved by augmenting the user-defined voice command set, also called grammar set, with alternate grammar expressions. For a given grammar set, a set of potential grammar expressions (candidate set) for augmentation is constructed from an AM-specific statistical pronunciation dictionary that captures the consistent patterns and errors in the decoding of AM induced by variations in pronunciation, pitch, tempo, accent, ambiguous spellings, and noise conditions. Using this candidate set, greedy optimization based and cross-entropy-method (CEM) based algorithms are considered to search for an augmented grammar set with improved recognition accuracy utilizing a command-specific dataset. Our experiments show that the proposed pipeline along with algorithms considered in this paper significantly reduce the mis-detection and mis-classification rate without increasing the false-alarm rate. Experiments also demonstrate the consistent superior performance of CEM method over greedy-based algorithms.

We consider the problem of communications over the binary symmetric channel with feedback, where the information sequence is made available in a causal, possibly random, fashion. We develop a real-time variant of the renowned Horstein scheme and provide analytical guarantees for its error-probability exponential decay rate. We further use the scheme to stabilize an unstable control plant over a binary symmetric channel and compare the analytical guarantees with its empirical performance as well as with those of anytime-reliable codes.

This paper considers the problem of bit-wise channel coding over a Binary Symmetric Channel (BSC) with feedback. While it is known that feedback does not increase the capacity of a memoryless channel, it is believed to simplify the coding schemes for some channels. The most significant one is the Binary Erasure Channel (BEC) where capacity is achieved by a simple sequential bit-wise repetition code under which each bit is (re-)transmitted until it is received. This sequential bit-wise feedback code has the added advantage that it can be used for streaming applications over BECs. In contrast, there is no known sequential bit-wise code with feedback that can achieve nonzero transmission rate for a BSC. For example, under Posterior Matching for a binary input channel with feedback, also known as Horstein scheme, each message is considered in its entirety in a block coding manner. This paper proposes a sequential feedback coding scheme with a nested bit-wise structure that generalizes repetition codes. This scheme is shown to achieve strictly positive rate for a large class of binary input channels including a BSC with arbitrary cross-over probability p ∈ (0, 1/2). The analysis relies on characterizing a lower bound on the step-wise Extrinsic Jensen Shannon divergence.

This paper considers the problem of searching for the unknown location of a target among a finite number of possible locations by probing multiple locations simultaneously. Outcome of each search measurement is corrupted by Gaussian noise whose intensity is proportional to the number of locations probed. We characterize a non-asymptotic lower bound on adaptivity gain; i.e. reduction in the expected number of measurements under an adaptive search strategies over the non-adaptive search strategies. Then we investigate the adaptivity gain in two complementary asymptotic regimes: one where the total search area is kept fixed but the location width is shrinking or the search resolution is increasing, and the other where each location width is fixed but the total search area is growing. Interestingly, adaptivity gain grows in distinctly different manner in these two regimes. In particular, adaptivity gains are significant in the later regime when the total search space grows; implying adaptivity is far more critical when either total search area or the noise intensity is large.

This paper considers a problem of distributed active hypothesis testing. At every time instant, individual nodes in the network adaptively choose a sensing action and receive noisy local (private) observations as sensing outcomes. The distribution of observations is parameterized by a discrete parameter (hypotheses). The marginals of the joint observation distribution conditioned on each hypothesis and the action are known locally at the nodes, but the true parameter/hypothesis is not known. An update rule is analyzed in which nodes first choose a possibly randomized action as a function of their past observations and actions. Nodes then perform a Bayesian update of their belief (distribution estimate) on each hypothesis based on their current local observations. Each node communicates these updates to its neighbors, and then performs a “non-Bayesian” linear consensus using the log-beliefs of its neighbors. Under mild assumptions and for a general class of action selection strategies, we show that the belief of any node on a wrong hypothesis converges to zero exponentially fast, and the exponential rate of learning is characterized by the nodes' influence of the network and average distinguishability between the observations' distributions for the (randomized) action under the true hypothesis.

The maximum type-I and type-II error exponents associated with the newly introduced almost-fixed-length hypothesis testing is characterized. In this class of tests, the decision-maker declares the true hypothesis almost always after collecting a fixed number of samples n; however in very rare cases with exponentially small probability the decision maker is allowed to collect another set of samples (no more than polynomial in n). This class of hypothesis tests are shown to bridge the gap between the classical hypothesis testing with a fixed sample size and the sequential hypothesis testing, and improve the trade-off between type-I and type-II error exponents.

This paper considers a problem of distributed hypothesis testing and social learning. Individual nodes in a network receive noisy (private) observations whose distribution is parameterized by a discrete parameter (hypotheses). The distributions are known locally at the nodes, but the true parameter/hypothesis is not known. An update rule is analyzed in which agents first perform a Bayesian update of their belief (distribution estimate) of the parameter based on their local observation, communicate these updates to their neighbors, and then perform a “non-Bayesian” linear consensus using the log-beliefs of their neighbors. The main result of this paper is that under mild assumptions, the belief of any agent in any incorrect parameter converges to zero exponentially fast, and the exponential rate of learning is a characterized by the network structure and the divergences between the observations' distributions.