Our work introduces a tile-aware scheduling framework for efficient sparse Vision Transformer execution on GPUs. Sparse attention reduces the cost of high-resolution Vision Transformers, but its irregular masks produce blocks with diverse sizes, densities, and locations. Existing FlashAttention-style kernels rely on fixed tile configurations and cannot fully exploit these sparse patterns, leading to wasted computation and underutilized GPU resources. Our framework bridges this gap through four key techniques: sparse attention is represented as an adjacency matrix, structure-aware reordering algorithms improve locality, locally dense blocks are extracted as scheduling units, and offline profiling with integer linear programming (ILP) selects hardware-feasible tile assignments. Results show that our inference scheduler achieves up to 2.13 × end-to-end speedup over fixed-tile FlashAttention and up to 4.6 × speedup in high-resolution images. We further introduce a training-aware extension that reuses the inference tile schedule and augments it with backward computation and activation-memory strategies.

Group convolutions are widely adopted in modern CNN architectures such as CondenseNet and ShuffleNet to enable efficient inference on GPUs. However, when the connection pattern between input and output channels does not exhibit regularity (unstructured group convolution), popular deep learning frameworks (e.g., PyTorch) often struggle with load balancing and data reuse issues leading to reduced performance. In this paper, we present a comprehensive optimization framework that combines a Knapsack-based partitioning approach with Integer Linear Programming (ILP) and advanced matrix reordering to optimize the deployment of unstructured group convolutions which are used in popular models such as CondenseNets that learn group connections. Specifically, we use knapsack algorithm to determine partition (group) sizes for the connections to minimize execution time and use an Integer Linear Programming (ILP) to assign connections to the partitions (group) output by the knapsack algorithm. We also employ three matrix reordering strategies-Hierarchical Clustering (HC), Iterative Clustering (IC), and Reverse Cuthill-McKee (RCM) on the matrix representing the input-output connectivity pattern to further improve the performance of our scheduling algorithm. Our experiments on ShuffleNet and CondenseNet demonstrate up to$1.9 \times$speedups over PyTorch. Furthermore, augmenting ILP with reordering achieves an additional$1.3 \times$improvement demonstrating the importance of optimizing for load balancing and data reuse.

Convolutional Neural Networks (CNN s) have demonstrated significant success in advancing image and video processing technologies, significantly outperforming traditional methods in both accuracy and efficiency. However, deploying CNN s effectively across diverse hardware platforms often faces the challenge of latency, which can critically impact real-time processing applications. In this work, we explore algorithmic design choices aimed at reducing latency in CNN deployments. We implement five convolution algorithms using SYCL and inte-grate them into three popular CNN models: VGG 16, Resnetl0l, and Inception V 4. By replacing the standard PyTorch Conv2d function with our SY CL- based implementations, we evaluate the execution time of each convolution layer and the overall model on G PU s. Our extensive experiments benchmark the performance of these algorithms against the baseline implementations of the PyTorch and Pytorch Extension for Intel. The results demonstrate significant improvements in execution time, underscoring the potential of these algorithmic choices for achieving low latency in CNN deployments.