We explore a strategy to handle controversial topics in LLM-based chatbots based on Wikipedia’s Neutral Point of View (NPOV) principle: acknowledge the absence of a single true answer and surface multiple perspectives. We frame this as retrieval augmented generation, where perspectives are retrieved from a knowledge base and the LLM is tasked with generating a fluent and faithful response from the given perspectives. As a starting point, we use a deterministic retrieval system and then focus on common LLM failure modes that arise during this approach to text generation, namely hallucination and coverage errors. We propose and evaluate three methods to detect such errors based on (1) word-overlap, (2) salience, and (3) LLM-based classifiers. Our results demonstrate that LLM-based classifiers, even when trained only on synthetic errors, achieve high error detection performance, with ROC AUC scores of 95.3% for hallucination and 90.5% for coverage error detection on unambiguous error cases. We show that when no training data is available, our other methods still yield good results on hallucination (84.0%) and coverage error (85.2%) detection.

Large language models have been shown to achieve remarkable performance across a variety of natural language tasks using few-shot learning, which drastically reduces the number of task-specific training examples needed to adapt the model to a particular application. To further our understanding of the impact of scale on few-shot learning, we trained a 540-billion parameter, densely activated, Transformer language model, which we call Pathways Language Model (PaLM). We trained PaLM on 6144 TPU v4 chips using Pathways, a new ML system which enables highly efficient training across multiple TPU Pods. We demonstrate continued benefits of scaling by achieving state-of-the-art few-shot learning results on hundreds of language understanding and generation benchmarks. On a number of these tasks, PaLM 540B achieves breakthrough performance, outperforming the finetuned state-of-the-art on a suite of multi-step reasoning tasks, and outperforming average human performance on the recently released BIG-bench benchmark. A significant number of BIG-bench tasks showed discontinuous improvements from model scale, meaning that performance steeply increased as we scaled to our largest model. PaLM also has strong capabilities in multilingual tasks and source code generation, which we demonstrate on a wide array of benchmarks. We additionally provide a comprehensive analysis on bias and toxicity, and study the extent of training data memorization with respect to model scale. Finally, we discuss the ethical considerations related to large language models and discuss potential mitigation strategies.

Scaling language models with more data, compute and parameters has driven significant progress in natural language processing. For example, thanks to scaling, GPT-3 was able to achieve strong results on in-context learning tasks. However, training these large dense models requires significant amounts of computing resources. In this paper, we propose and develop a family of language models named \glam (\textbf{G}eneralist \textbf{La}nguage \textbf{M}odel), which uses a sparsely activated mixture-of-experts architecture to scale the model capacity while also incurring substantially less training cost compared to dense variants. The largest \glam has 1.2 trillion parameters, which is approximately 7x larger than GPT-3. It consumes only 1/3 of the energy used to train GPT-3 and requires half of the computation flops for inference, while still achieving better overall fewshot performance across 29 NLP tasks.

The Domain Name System (DNS) is an essential network infrastructure component since it supports the operation of the Web, Email, Voice over IP (VoIP) and other business- critical applications running over the network. Events that compromise the security of DNS can have a significant impact on the Internet since they can affect its availability and its intended operation. This paper describes algorithms used to monitor and detect certain types of attacks to the DNS infrastructure using flow data. Our methodology is based on algorithms that do not rely on known signature attack vectors. The effectiveness of our solution is illustrated with real and simulated traffic examples. In one example, we were able to detect a tunneling attack well before the appearance of public reports of it.

At the switch boundaries of cellular systems, it is necessary to provide an inter-switch handover procedure if calls are to remain in progress as users cross these boundaries. Both the North American and Pan European second generation standards provide mechanisms for inter-switch handover. Motivated by the need to develop procedures for inter-switch handover in the cellular packet switch (CPS), this paper develops a general set of procedures for inter-switch handover that, unlike their second generation counterparts, provide a mechanism for the transport of data. Unlike voice, data packets must all be delivered. The delivery may need to be sequential and/or restricted to a single copy. The path optimization procedures are all presented in the context of the CPS. In addition, we identify capabilities needed by the CPS and present suitable protocols. The path optimization procedures are evaluated based on the following performance criteria: guarantee of sequential delivery, avoidance of multiple copies, and delay. Fundamental limitations on data throughput result if both sequential delivery and a single copy are required.

Future wireless information networks will employ dense grids of base stations to support a high user population. While small cells mitigate capacity problems, lengthen battery life, and improve signal quality, the frequent movement of users across cell boundaries imposes new burdens on network controllers. Previous work suggests that a cellular packet switch, based on distributed network control, can support high density personal communications. This paper proposes an allocation of control functions to elements of the cellular packet switch and specifies the flow of information within the switch. Message flows, defined for key procedures, form the foundation of a network layer protocol. Functionality is divided into the categories of call processing (set up and release), mobility management (location updates, handover, and path optimization), and radio resource management. Central office switches perform call processing. Mobility management and radio resource management are performed by interface units of the metropolitan area network at the heart of the cellular packet switch. This architecture delivers advanced services with minimal impact on central office switch software. It also relies on distributed, rather than centralized mobility management to provide prompt, accurate communications as users move about.>

A method for packet-switch provisioning which accounts for the effects of uncertainty in estimating customer characteristics is demonstrated. The method models the traffic parameters of the customers as random variables. It is used to calculate the maximum number of customers that can be accommodated while controlling the probability that their traffic control is beyond the nominal capacity of the system. It is shown that, depending on the degree of uncertainty in estimating parameters for the arrival process, the application of the method can have a substantial impact on the provisioning of the switch. Examples in which the effects of uncertainty on engineering are quantified are given.>

A methodology is presented for analyzing a queuing submodel which frequently arises in the study of overflow models. In this submodel a finite capacity, multiserver queue with exponentially distributed service times, and arriving traffic consisting of a Poisson parcel and several overflow parcels, are assumed. By modeling the overflow parcels as interrupted Poisson processes, an exact queuing analysis is possible. The analysis yields the steady-state queue length distribution, and for each input parcel: (1) the steady-state queue length distribution at arrivals; (2) the probability that an arriving call is blocked (parcel blocking); and (3) the waiting time distribution of an arriving call, in addition to a complete characterization of the overflow due to each parcel.>