We present LINGUIST, a method for generating annotated data for Intent Classification and Slot Tagging (IC+ST), via fine-tuning AlexaTM 5B, a 5-billion-parameter multilingual sequence-to-sequence (seq2seq) model, on a flexible instruction prompt. In a 10-shot novel intent setting for the SNIPS dataset, LINGUIST surpasses state-of-the-art approaches (Back-Translation and Example Extrapolation) by a wide margin, showing absolute improvement for the target intents of +1.9 points on IC Recall and +2.5 points on ST F1 Score. In the zero-shot cross-lingual setting of the mATIS++ dataset, LINGUIST out-performs a strong baseline of Machine Translation with Slot Alignment by +4.14 points absolute on ST F1 Score across 6 languages, while matching performance on IC. Finally, we verify our results on an internal large-scale multilingual dataset for conversational agent IC+ST and show significant improvements over a baseline which uses Back-Translation, Paraphrasing and Slot Catalog Resampling. To our knowledge, we are the first to demonstrate instruction fine-tuning of a large-scale seq2seq model to control the outputs of multilingual intent- and slot-labeled data generation.

We present results from a large-scale experiment on pretraining encoders with non-embedding parameter counts ranging from 700M to 9.3B, their subsequent distillation into smaller models ranging from 17M-170M parameters, and their application to the Natural Language Understanding (NLU) component of a virtual assistant system. Though we train using 70% spoken-form data, our teacher models perform comparably to XLM-R and mT5 when evaluated on the written-form Cross-lingual Natural Language Inference (XNLI) corpus. We perform a second stage of pretraining on our teacher models using in-domain data from our system, improving error rates by 3.86% relative for intent classification and 7.01% relative for slot filling. We find that even a 170M-parameter model distilled from our Stage 2 teacher model has 2.88% better intent classification and 7.69% better slot filling error rates when compared to the 2.3B-parameter teacher trained only on public data (Stage 1), emphasizing the importance of in-domain data for pretraining. When evaluated offline using labeled NLU data, our 17M-parameter Stage 2 distilled model outperforms both XLM-R Base (85M params) and DistillBERT (42M params) by 4.23% to 6.14%, respectively. Finally, we present results from a full virtual assistant experimentation platform, where we find that models trained using our pretraining and distillation pipeline outperform models distilled from 85M-parameter teachers by 3.74%-4.91% on an automatic measurement of full-system user dissatisfaction.

Semantic parsing is one of the key components of natural language understanding systems.A successful parse transforms an input utterance to an action that is easily understood by the system.Many algorithms have been proposed to solve this problem, from conventional rulebased or statistical slot-filling systems to shiftreduce based neural parsers.For complex parsing tasks, the state-of-the-art method is based on autoregressive sequence to sequence models to generate the parse directly.This model is slow at inference time, generating parses in O(n) decoding steps (n is the length of the target sequence).In addition, we demonstrate that this method performs poorly in zero-shot cross-lingual transfer learning settings.In this paper, we propose a non-autoregressive parser which is based on the insertion transformer to overcome these two issues.Our approach 1) speeds up decoding by 3x while outperforming the autoregressive model and 2) significantly improves cross-lingual transfer in the low-resource setting by 37% compared to autoregressive baseline.We test our approach on three well-known monolingual datasets: ATIS, SNIPS and TOP.For cross lingual semantic parsing, we use the MultiATIS++ and the multilingual TOP datasets.

We introduce and study the doubly balanced connected graph partitioning problem: Let G =( V , E ) be a connected graph with a weight (supply/demand) function p : V → {−1, +1} satisfying p ( V )=∑ j &isin V p ( j ) = 0. The objective is to partition G into ( V 1 , V 2 ) such that G [ V 1 ] and G [ V 2 ] are connected, ∣ p ( V 1 )∣,∣ p ( V 2 )∣≤ c p , and max{ ∣ V 1 / V 2 ∣,∣ V 2 / V 1 ∣} ≤ c s , for some constants c p and c s . When G is 2-connected, we show that a solution with c p =1 and c s =2 always exists and can be found in randomized polynomial time. Moreover, when G is 3-connected, we show that there is always a “perfect” solution (a partition with p ( V 1 )= p ( V 2 )=0 and ∣ V 1 ∣=∣ V 2 ∣, if ∣ V ∣≡ 0 (mod 4)), and it can be found in randomized polynomial time. Our techniques can be extended, with similar results, to the case in which the weights are arbitrary (not necessarily ±1), and to the case that p ( V )≠ 0 and the excess supply/demand should be split evenly. They also apply to the problem of partitioning a graph with two types of nodes into two large connected subgraphs that preserve approximately the proportion of the two types.

We introduce and study the Doubly Balanced Connected graph Partitioning (DBCP) problem: Let G=(V, E) be a connected graph with a weight (supply/demand) function p:V→{-1, +1} satisfying P(V)=∑j∊v p(j)=0· The objective is to partition G into (V1, V2) such that G[V1] and G[V2] are connected, |p(V1)|, |p(V2)|≤cp, and for some constants cp and cs · When G is 2-connected, we show that a solution with cp=1 and cs=3 always exists and can be found in polynomial time. Moreover, when G is 3-connected, we show that there is always a ‘perfect’ solution (a partition with p(V1)=p(V2)=0 and |V1| = |V2|, if |V|≡0(mod 4)), and it can be found in polynomial time. Our techniques can be extended, with similar results, to the case in which the weights are arbitrary (not necessarily ±1), and to the case that p(V)=0 and the excess supply/demand should be split evenly. They also apply to the problem of partitioning a graph with two types of nodes into two large connected subgraphs that preserve approximately the proportion of the two types.

Recent events demonstrated the vulnerability of power grids to cyber attacks and to physical attacks. Therefore, we focus on joint cyber and physical attacks and develop methods to retrieve the grid state information following such an attack. We consider a model in which an adversary attacks a zone by physically disconnecting some of its power lines and blocking the information flow from the zone to the grid's control center. We use tools from linear algebra and graph theory and leverage the properties of the power flow DC approximation to develop methods for information recovery. Using information observed outside the attacked zone, these methods recover information about the disconnected lines and the phase angles at the buses. We identify sufficient conditions on the zone structure and constraints on the attack characteristics such that these methods can recover the information. We also show that it is NP-hard to find an approximate solution to the problem of partitioning the power grid into the minimum number of attack-resilient zones. However, since power grids can often be represented by planar graphs, we develop a constant approximation partitioning algorithm for these graphs. Finally, we numerically study the relationships between the grid's resilience and its structural properties, and demonstrate the partitioning algorithm on real power grids. The results can provide insights into the design of a secure control network for the smart grid.

This paper focuses on cascading line failures in the transmission system of the power grid. Such a cascade may have a devastating effect not only on the power grid but also on the interconnected communication networks. Recent large-scale power outages demonstrated the limitations of epidemic- and percolation-based tools in modeling the cascade evolution. Hence, based on a linearized power flow model (that substantially differs from the classical packet flow models), we obtain results regarding the various properties of a cascade. Specifically, we consider performance metrics such as the the distance between failures, the length of the cascade, and the fraction of demand (load) satisfied after the cascade. We show, for example, that due to the unique properties of the model: (i) the distance between subsequent failures can be arbitrarily large and the cascade may be arbitrarily long, (ii) a large set of initial line failures may have a smaller effect than a failure of one of the lines in the set, and (iii) minor changes to the network parameters may have a significant impact. Moreover, we show that finding the set of lines whose removal has the most significant impact (under various metrics) is NP-Hard. Moreover, we develop a fast algorithm to recompute the flows at each step of the cascade. The results can provide insight into the design of smart grid measurement and control algorithms that can mitigate a cascade.