Multi-headed attention heads are a mainstay in transformer-based models. Different methods have been proposed to classify the role of each attention head based on the relations between tokens which have high pair-wise attention. These roles include syntactic (tokens with some syntactic relation), local (nearby tokens), block (tokens in the same sentence) and delimiter (the special [CLS], [SEP] tokens). There are two main challenges with existing methods for classification: (a) there are no standard scores across studies or across functional roles, and (b) these scores are often average quantities measured across sentences without capturing statistical significance. In this work, we formalize a simple yet effective score that generalizes to all the roles of attention heads and employs hypothesis testing on this score for robust inference. This provides us the right lens to systematically analyze attention heads and confidently comment on many commonly posed questions on analyzing the BERT model. In particular, we comment on the co-location of multiple functional roles in the same attention head, the distribution of attention heads across layers, and effect of fine-tuning for specific NLP tasks on these functional roles. The code is made publicly available at https://github.com/iitmnlp/heads-hypothesis

Given the success of Transformer-based models, two directions of study have emerged: interpreting role of individual attention heads and down-sizing the models for efficiency.Our work straddles these two streams: We analyse the importance of basing pruning strategies on the interpreted role of the attention heads.We evaluate this on Transformer and BERT models on multiple NLP tasks.Firstly, we find that a large fraction of the attention heads can be randomly pruned with limited effect on accuracy.Secondly, for Transformers, we find no advantage in pruning attention heads identified to be important based on existing studies that relate importance to the location of a head.On the BERT model too we find no preference for top or bottom layers, though the latter are reported to have higher importance.However, strategies that avoid pruning middle layers and consecutive layers perform better.Finally, during fine-tuning the compensation for pruned attention heads is roughly equally distributed across the un-pruned heads.Our results thus suggest that interpretation of attention heads does not strongly inform pruning.