Vision Transformers have been proven powerful in various vision applications. Yet, their adaptations for video understanding tasks incur large computational costs, limiting their practical deployment on resource-constrained devices. Token pruning can effectively alleviate the processing overhead of underlying attention blocks, but often neglects the iterative processing nature of video models applied frame-by-frame. We propose to prune tokens according to the estimated contribution of their corresponding tokens in previous frames to previous predictions. We leverage attention rollout and token tracking to propagate token importance of previous outputs to current input tokens. Our method is interpretable, requires no training and has negligible memory overhead. We show the efficacy of our method for both video object detection and action recognition using different transformer architectures, achieving up to 65% reduction in FLOPS on ImageNet VID and 60% on EPIC-Kitchens with no accuracy degradation. We release the code and models at https://github.com/RGTPdyn/RGTP.

Network-Based Biocomputation (NBC) circuits are computational devices that utilize biological agents to efficiently explore designed nanofabricated networks and thus solve combinatorial problems. The main advantages of NBCs are the potential to combine massively parallel computation, inherent energy efficiency of the biological agents and maturity of nanofabrication technology. We present an integrated computational-aided toolset for simulation and verification of these circuits that enables analysis of both circuit correctness and the effects of agent stochastic dynamics on circuit behavior. Our approach enables early identification of design flaws and can lead to significant savings in resources, thus playing an important role in advancing this emerging paradigm.

Abstract Network-Based Biocomputation Circuits (NBCs) offer a new paradigm for solving complex computational problems by utilizing biological agents that operate in parallel to explore manufactured planar devices. The approach can also have future applications in diagnostics and medicine by combining NBCs computational power with the ability to interface with biological material. To realize this potential, devices should be designed in a way that ensures their correctness and robust operation. For this purpose, formal methods and tools can offer significant advantages by allowing investigation of design limitations and detection of errors before manufacturing and experimentation. Here we define a computational model for NBCs by providing formal semantics to NBC circuits. We present a formal verification-based approach and prototype tool that can assist in the design of NBCs by enabling verification of a given design’s correctness. Our tool allows verification of the correctness of NBC designs for several NP-Complete problems, including the Subset Sum, Exact Cover and Satisfiability problems and can be extended to other NBC implementations. Our approach is based on defining transition systems for NBCs and using temporal logic for specifying and proving properties of the design using model checking. Our formal model can also serve as a starting point for computational complexity studies of the power and limitations of NBC systems.