Recent advancements in computer vision, including convolutional neural networks, multilayer perceptrons, graph-based methods and transformer architectures, have significantly improved image classification. However, applying these techniques to digital pathology, particularly gigapixel whole-slide images (WSIs), presents unique challenges due to their vast size and heterogeneity. We introduce FourierMIL, a multiple instance learning framework that leverages the discrete Fourier transform to efficiently capture global and local dependencies in WSIs. Unlike conventional approaches, FourierMIL is adaptable to diverse digital stains and pathology tasks. To evaluate its versatility, we tested FourierMIL on three distinct challenges using public and private datasets. (1) Metastasis detection in hematoxylin and eosin (H&E)- stained lymph node WSIs from CAncer MEtastases in LYmph nOdes challeNge (CAMELYON16) dataset. (2) Lung cancer classification (adenocarcinoma versus squamous cell carcinoma) using The Cancer Genome Atlas (TCGA) and the Clinical Proteomic Tumor Analysis Consortium (CPTAC) datasets. (3) Alzheimer's disease pathology identification in phospho-tau monoclonal antibody (AT8)- stained WSIs from the Understanding Neurologic Injury and Traumatic Encephalopathy (UNITE), the Framingham Heart Study (FHS), and the Boston University Alzheimer's Disease Research Center (ADC) cohorts. FourierMIL outperformed state-of-the-art methods across all tasks, demonstrating its robustness as an attention-free solution for diverse applications in digital pathology.