Abstract We evaluate recent Large Language Models (LLMs) on the challenging task of summarizing short stories, which can be lengthy, and include nuanced subtext or scrambled timelines. Importantly, we work directly with authors to ensure that the stories have not been shared online (and therefore are unseen by the models), and to obtain informed evaluations of summary quality using judgments from the authors themselves. Through quantitative and qualitative analysis grounded in narrative theory, we compare GPT-4, Claude-2.1, and LLama-2-70B. We find that all three models make faithfulness mistakes in over 50% of summaries and struggle with specificity and interpretation of difficult subtext. We additionally demonstrate that LLM ratings and other automatic metrics for summary quality do not correlate well with the quality ratings from the writers.

Today there exists no shortage of outlier detection algorithms in the literature, yet the complementary and critical problem of unsupervised outlier model selection (UOMS) is vastly understudied. In this work, we propose ELECT, a new approach to select an effective candidate model, i.e. an outlier detection algorithm and its hyperparameter(s), to employ on a new dataset without any labels. At its core, ELECT is based on meta-learning; transferring prior knowledge (e.g. model performance) on historical datasets that are similar to the new one to facilitate UOMS. Uniquely, it employs a dataset similarity measure that is performance-based, which is more direct and goal-driven than other measures used in the past. ELECT adaptively searches for similar historical datasets, as such, it can serve an output on-demand, being able to accommodate varying time budgets. Extensive experiments show that ELECT significantly outperforms a wide range of basic UOMS baselines, including no model selection (always using the same popular model such as iForest) as well as more recent selection strategies based on meta-features.

There is no shortage of outlier detection (OD) algorithms in the literature, yet a vast body of them are designed for a single machine. With the increasing reality of already cloud-resident datasets comes the need for distributed OD techniques. This area, however, is not only understudied but also short of public-domain implementations for practical use. This paper aims to fill this gap: We design Sparx, a data-parallel OD algorithm suitable for shared-nothing infrastructures, which we specifically implement in Apache Spark. Through extensive experiments on three real-world datasets, with several billions of points and millions of features, we show that existing open-source solutions fail to scale up; either by large number of points or high dimensionality, whereas Sparx yields scalable and effective performance. To facilitate practical use of OD on modern-scale datasets, we open-source Sparx under the Apache license at https://tinyurl.com/sparx2022.

Conventional radar networks are limited at warning against low-altitude wind hazards such as tornadoes and micro-bursts due to the combined effects of their long range, far spacing, and Earth's curvature. The Collaborative Adaptive Sensing of the Atmosphere (CASA) Engineering Research Center aims to solve these limitations through dual-Doppler systems operating in densely-spaced short range radar networks. The CASA test bed radar network incorporate the Distributed Collaborative Adaptive Sensing (DCAS) model in order to optimize scanning and retrieval for fast Doppler wind field products. Under the DCAS environment, fast coordinated sector scans are made by each radar based on weather detection and competitive end-user needs. The retrieval subsystem then makes the best pair selection for dual-Doppler synthesis based on optimal beam-crossing angles for target areas. Together, they create a fast, accurate, and robust dual-Doppler system suitable for weather emergency warnings.

The first interprocedural modification side-effects analysis for C (MOD C ) that obtains better than worst-case precision on programs with general-purpose pointer usage is presented with empirical results. The analysis consists of an algorithm schema corresponding to a family of MOD C algorithms with two independent phases: one for determining pointer-induced aliases and a subsequent one for propagating interprocedural side effects. These MOD C algorithms are parameterized by the aliasing method used. The empirical results compare the performance of two dissimilar MOD C algorithms: MOD C ( FSAlias ) uses a flow-sensitive, calling-context-sensitive interprocedural alias analysis; MOD C ( FIAlias uses a flow-insensitive, calling-context-insensitive alias analysis which is much faster, but less accurate. These two algorithms were profiled on 45 programs ranging in size from 250 to 30,000 lines of C code, and the results demonstrate dramatically the possible cost-precision trade-offs. This first comparative implementation of MOD C analyses offers insight into the differences between flow-/context-sensitive and flow-/context-insensitive analyses. The analysis cost versus precision trade-offs in side-effect information obtained are reported. The results show surprisingly that the precision of flow-sensitive side-effect analysis is not always prohibitive in cost, and that the precision of flow-insensitive analysis is substantially better than worst-case estimates and seems sufficient for certain applications. On average MOD C ( FSAlias ) for procedures and calls is in the range of 20% more precise than MOD C ( FIAlias ); however, the performance was found to be at least an order of magnitude slower than MOD C ( FIAlias ).

Precision and scalability are two desirable, yet often conflicting, features of data-flow analyses. This paper reports on a case study of the modification---ide-effects problem for C in the presence of pointers from the perspective of contrasting the flow and context sensitivity of the solution procedure with respect to precision and scalability. The results show that the cost of precision of flow- and context-sensitive analysis is not always prohibitive, and that the precision of flow- and context-insensitive analysis is substantially better than worst-case estimates and can be sufficient for certain applications. Program characteristics that affect the performance of data-flow analysis are identified.

We present initial empirical experiments with combined analysis, a scalabk analysis technique that uses a program decomposition to apply different aliasing algorithms to independent program segments. The effectiveness of the solution strategy is validated through application to side-effect and reference analysis of C programs.

Pointer aliasing analysis is crucial to compile-time analyses for languages with general-purpose pointer usage (such as C), but many aliasing methods have proven quite costly. We present a technique that partitions the statements of a program to allow separate, and therefore possibly different, pointer aliasing analysis methods to be used on independent parts of the program. This decomposition enables exploration of tradeoff between algorithm efficiency and precision. We also present a new, efficient flow-insensitive pointer aliasing algorithm, which is used together with an existing flow-sensitive aliasing algorithm in our experiments. We demonstrate our technique in the context of determining side effects and variable fetches through names containing pointer dereferences (Thru-deref MOD/REF). Initial empirical results using a combination of a flow-sensitive and a flow-insensitive aliasing analysis on the same program, demonstrate that the resulting analysis is much faster than solely using the flow-sensitive method, and obtains similar precision for the Thru-deref MOD/REF problems.

We present a new interprocedural modification side effects algorithm for C programs, that can discern side effects through general-purpose pointer usage. Ours is the first complete design and implementation of such an algorithm. Preliminary performance findings support the practicality of the technique, which is based on our previous approximation algorithm for pointer aliases [LR92]. Each indirect store through a pointer variable is found, on average, to correspond to a store into 1.2 locations. This indicates that our program-point-specific pointer aliasing information is quite precise when used to determine the effects of these stores.