one setup is more effective than another. In this work, we uncover the underlying characteristics of effective sets. For each training instance, we form a large, diverse pool of candidates and systematically vary the subsets used for calibration fine-tuning. Each selection strategy targets distinct aspects of the sets, such as lexical diversity or the size of the gap between positive and negatives. On three diverse scientific long-form summarization datasets (spanning biomedical, clinical, and chemical domains), we find, among others, that faithfulness calibration is optimal when the negative sets are extractive and more likely to be generated, whereas for relevance calibration, the metric margin between candidates should be maximized and surprise-the disagreement between model and metric defined candidate rankings-minimized. Code to create, select, and optimize calibration sets is available at https://github.com/griff4692/calibrating-summaries.

Yixin Liu, Budhaditya Deb, Milagro Teruel, Aaron Halfaker, Dragomir Radev, Ahmed Hassan Awadallah. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2023.

Yusen Zhang, Ansong Ni, Ziming Mao, Chen Henry Wu, Chenguang Zhu, Budhaditya Deb, Ahmed Awadallah, Dragomir Radev, Rui Zhang. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2022.

Ziming Mao, Chen Henry Wu, Ansong Ni, Yusen Zhang, Rui Zhang, Tao Yu, Budhaditya Deb, Chenguang Zhu, Ahmed Awadallah, Dragomir Radev. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2022.

Recent work has shown that language models (LMs) trained with multi-task instructional learning (MTIL) can solve diverse NLP tasks in zero-and few-shot settings with improved performance compared to prompt tuning.MTIL illustrates that LMs can extract and use information about the task from instructions beyond the surface patterns of the inputs and outputs.This suggests that meta-learning may further enhance the utilization of instructions for effective task transfer.In this paper we investigate whether meta-learning applied to MTIL can further improve generalization to unseen tasks in a zero-shot setting.Specifically, we propose to adapt meta-learning to MTIL in three directions: 1) Model Agnostic Meta Learning (MAML), 2) Hyper-Network (HNet) based adaptation to generate task specific parameters conditioned on instructions, and 3) an approach combining HNet and MAML.Through extensive experiments on the large scale Natural Instructions V2 dataset, we show that our proposed approaches significantly improve over strong baselines in zero-shot settings.In particular, meta-learning improves the effectiveness of instructions and is most impactful when the test tasks are strictly zero-shot (i.e.no similar tasks in the training set) and are "hard" for LMs, illustrating the potential of meta-learning for MTIL for out-of-distribution tasks.

Neural attention models have achieved significant improvements on many natural language processing tasks.However, the quadratic memory complexity of the self-attention module with respect to the input length hinders their applications in long text summarization.Instead of designing more efficient attention modules, we approach this problem by investigating if models with a restricted context can have competitive performance compared with the memory-efficient attention models that maintain a global context by treating the input as a single sequence.Our model is applied to individual pages, which contain parts of inputs grouped by the principle of locality, during both the encoding and decoding stages.We empirically investigated three kinds of locality in text summarization at different levels of granularity, ranging from sentences to documents.Our experimental results show that our model has a better performance compared with strong baseline models with efficient attention modules, and our analysis provides further insights into our locality-aware modeling strategy.1

Mozhi Zhang, Wei Wang, Budhaditya Deb, Guoqing Zheng, Milad Shokouhi, Ahmed Hassan Awadallah. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.

Periodic transitions from place to place are inherent in human movements. Through visual examination we detect these periodic movements in traces of user tracking data. However such user tracking data sets tend to be sparse and incomplete. In addition, periodic movements are surrounded by noise: transitions to and from less frequently visited places and transitions to one of a kind visits. In this paper, we present algorithms leveraging techniques and models to detect periodicity in individual user movements. Our algorithms predict a user's next place given only the current context of timestamp and location. We apply these algorithms to real user mobility data sets. Prediction accuracy depends on the ratio of periodic movements to noise in user traces. For majority of users in a movement tracking data set collected over a year, our algorithms achieve next place prediction accuracies of 50% and above.

We propose a distributed algorithm for optimizing the Contention Windows in IEEE 802.11e based WLANs with the dual intention of providing fine-grained QoS and maximizing the channel utilization. The underlying concept behind the algorithm is modeling the network state as a function of MAC parameters and solving this analytical model constrained by the QoS requirements of multiple nodes. The main contribution of this paper is the completely distributed realization of this concept. The problem appears as a system of non-linear equations which is solved by an iterative gradient-based method. Distributed solution is achieved by first decoupling the equations and second by implicit message passing through local measurements. This allows local computation of partial differentials and residuals of the iterative process. Local measurements serve as inputs for the next iterative step and as natural feedback mechanism to handle dynamic channel conditions. Convergence of iterations is ensured through progressive target setting of QoS requirements. Extensive simulation results and a proof of concept with a test bed show that the algorithm achieves fine-grained QoS differentiation while minimizing delays, collisions and packet losses. As a result, when the network scales, the algorithm is shown to maximize the channel utilization and maintain a near optimal total throughput of the system. Finally, sub-minute convergence time makes the algorithm suitable for real-time flows.

In this paper, we analyze the node scheduling approach of topology control in the context of reliable packet delivery. In node scheduling, only a minimum set of nodes needed for routing purposes (usually determined by a minimum connected dominating set, MCDS) are kept active. However, a very low density resulting from switching off nodes can adversely affect the performance of data delivery due to three factors. First, our analysis shows that at low density, the average path length increases by a factor more than previously thought. Second, protocols such as the Hop-By-Hop Broadcast (HHB) reliability scheme (which relies on high network degree for optimum performance) suffer. Third, with limited buffers at nodes, the overhead is more pronounced to the extent of making the network unstable. Using probabilistic models, we derive the relationship between network density and overhead based on the above factors and find the density conditions for minimum power consumption. We also propose a, fully distributed and message-optimal node scheduling algorithm with a constant approximation bound based on the concept of Virtual Connected Dominating Sets. The scheme can asymptotically achieve optimal density conditions while adapting to different network parameters.

Sensor networks are meant for sensing and disseminating information about the environment they sense. The criticality of a sensed phenomenon determines its importance to the end user. Hence, data dissemination in a sensor network should be information aware. Such information awareness is essential firstly to disseminate critical information more reliably and secondly to consume network resources proportional to the criticality of information. In this paper, we describe a protocol called RelnForM to deliver packets at desired reliability at a proportionate communication cost. RelnForm sends multiple copies of each packet along multiple paths from source to sink, such that data is delivered at the desired reliability. It uses the concept of dynamic packet state in context of sensor networks, to control the number of paths required for the desired reliability, and does so using only local knowledge of channel error rates and topology. We show that for uniform unit disk graphs, the number of edge-disjoint paths between nodes is equal to the average node degree with very high probability. RelnForm utilizes this property in its randomized forwarding mechanism which results in use of all possible paths and efficient load balancing.