Modern Automatic Speech Recognition (ASR) systems are evaluated with respect to Word Error Rate (WER). While WER is a useful metric for training and evaluation of speech models, it does not fully reflect the difference in semantics between predicted and ground truth transcriptions. In conversational voice assistants, the ability to sufficiently understand semantic meaning of the user request is often more important than recognizing all words correctly. In this work, we propose a system that can determine, to a high degree of accuracy, whether the semantics of a predicted and reference transcript are significantly different. This knowledge is used to identify ASR errors that can result in downstream failure in conversational voice assistants. Reliable identification of these errors can be used to inform design choices for ASR systems targeting improvement on the most harmful errors.

The focus of this paper is a Bayesian framework for solving a class of problems termed multiagent inverse reinforcement learning (MIRL). Compared to the well-known inverse reinforcement learning (IRL) problem, MIRL is formalized in the context of stochastic games, which generalize Markov decision processes to game theoretic scenarios. We establish a theoretical foundation for competitive two-agent zero-sum MIRL problems and propose a Bayesian solution approach in which the generative model is based on an assumption that the two agents follow a minimax bipolicy. Numerical results are presented comparing the Bayesian MIRL method with two existing methods in the context of an abstract soccer game. Investigation centers on relationships between the extent of prior information and the quality of learned rewards. Results suggest that covariance structure is more important than mean value in reward priors.

We consider the problem of transmitting a collection of packets from a source node to a destination node across a relay network. We analyze a simple random network coding scheme where each node transmits a random linear combination of packets each time it has an opportunity to transmit. The main result of this paper is an upper bound on the expected time to transmit a generation of packets across the network. We show that the expected time is bounded by the generation size divided by the capacity of the minimum cut separating the source from the destination, plus a term that grows as the square root of the generation size. We then use this bound to provide a queueing analysis of a strategy that dynamically creates generations as packets arrive in a queue at the network's source node. To facilitate our analysis, we model the relay network by a continuous-time Markov chain. Our primary analytical tool is a general method for computing upper bounds on hitting times associated with continuous-time Markov chains. We believe that this approach also provides a method for analyzing transmission times associated with more general network topologies.

Graph similarity measurement has been used in many applications, such as computational biology, text mining, pattern recognition, and computer vision. In this paper, we apply similarity measurement on graphs to measure structural differences in financial statements. Unconventional financial statement structures may potentially reveal deceptive intention of hiding certain information while making technically “correct” financial statements. Furthermore, unconventional financial statements may also lead to investment opportunities if legitimacy is not questioned. We construct an algorithm based on the metric of string edit distance as an approximation of graph similarity, and apply the Levenshtein algorithm with modified string edit costs to measure string edit distance. We demonstrate the effectiveness of this algorithm in capturing the sensitive changes of balance sheet structures by applying the algorithm in two experiments. The first experiment shows the algorithm is sensitive to all three basic edits (namely deletion, insertion and substitution) on a particular balance sheet, and the second experiment shows more than 90% clustering accuracy on real balance sheets.

Due to the NP-hardness of factor-graph optimization, obtaining exact solutions to problems with a large variable domain is generally not possible. Max-sum (max-product) belief propagation (BP) is a distributed message-passing heuristic that has found popularity due to its ability to generate approximate solutions to such factor-graph problems in a distributed fashion. Because max-sum BP generally provides no indication of solution quality, researchers have sought alternative algorithms to generate approximate (and, in some cases, exact) solutions, the most successful of which operate on a relaxation of the integer programming form of an equivalent maximum a posteriori estimation problem. While such linear-programming-based algorithms perform well in empirical studies, there are limits to the variable domain size for which they are tractable. Via a case study in weather-radar coordination, we demonstrate that the decentralized max-sum BP algorithm remains useful for generating quality solutions to problems with a large variable domain. Our custom simulation tool facilitates a comparison of the performance of algorithms with respect to adaptive weather-radar scanning resource allocation across three weather scenarios. In addition to no adaptive scanning, the algorithms include four max-sum-BP-based algorithms: decentralized distributed max-sum BP, self-terminating tree-based bounded approximation, tabu search implemented in a centralized fashion, and a combination of the latter two. Performance is measured by the end-user utility for all algorithms and by two types of approximation ratios for the tree-based bounded approximation. BP-based decentralized algorithms are found to exhibit comparable performance with a centralized algorithm and superior performance to no adaptive scanning. Furthermore, our analysis demonstrates that max-sum BP is capable of generating solutions within 67% of optimal (and typically much better) across the weather scenarios.

In this work we analyze the average queue backlog for transmission of a single multicast flow consisting of M destination nodes in a wireless network. In the model we consider, the channel between every pair of nodes is an independent identically distributed packet erasure channel. We first develop a lower bound on the average queue backlog achievable by any transmission strategy; for a single-hop multicast transmission, our bound indicates that the queue size must scale as at least Ω(ln(M)). Next, we generalize this result to a multihop network and obtain a lower bound on the queue backlog as it relates to the minimum-cut capacity of the network. We then analyze the queue backlog for a strategy in which random linear coding is performed over groups of packets in the queue at the source node of a single-hop multicast. We develop an upper bound on the average queue backlog for the packet-coding strategy to show that the queue size for this strategy scales as O(ln(M)). Our results demonstrate that in terms of the queue backlog for single-hop multicast, the packet coding strategy is order-optimal with respect to the number of receivers.

This paper compares scheduling and coding strategies for a multicast version of a classic downlink problem. We consider scheduling strategies where, in each time slot, a scheduler observes the lengths of all queues and the connectivities of all links and can transmit the head-of-the-line packet from a single queue. We juxtapose this to a coding strategy that is simply a form of classical random linear coding. We show that there are configurations for which the stable throughput region of the scheduling strategy is a strict subset of the corresponding throughput region of the coding strategy. This analysis is performed for both time-invariant and time-varying channels. The analysis is also performed both with and without accounting for the impact on throughput of including coding overhead symbols in each encoded packet. Additionally, we compare coding strategies that only code within individual queues against a coding strategy that codes across separate queues. The strategy that codes across queues simply sends packets from all queues to all receivers. As a result, this strategy sends many packets to unnecessary recipients. We show, surprisingly, that there are cases where the strategy that codes across queues can achieve the same throughput region achievable by coding within individual queues.

In this paper we consider the problem of computing the expected hitting time to a vertex for random walks on graphs. We give a method for computing an upper bound on the expected hitting time from an arbitrary spanning tree of the graph. We illustrate this method with two examples. In one of these examples, we show that the bounds obtained from the spanning method are sharper than bounds obtained from other commonly used techniques.

We consider a problem of managing separate multicast sessions from a single transmitter. Each of K sessions has an associated packet stream, and a single transmitter must transmit these packet streams to a group of receivers. The multicast sessions are separate in the sense that each receiver only wants packets from one of the K streams. We will compare the maximum stable arrival rates that can be supported with and without using random linear coding across the K sessions. Intuitively, it seems that coding across sessions is not beneficial. Coding across sessions appears to introduce unnecessary additional delay since each receiver does not receive its next packet until it can decode the head-of-line packets from all K streams. However, we show that in many cases the maximum stable arrival rate that can be supported when coding across sessions is significantly greater than maximum stable arrival rate that can be supported when not coding across sessions. We provide a sufficient condition that indicates when coding across sessions is preferable. This condition is expressed in terms of the number of sessions, the number of receivers per session, and the reliability of the channels connecting the transmitter to the receivers.