Test-time adaptation allows pretrained models to adjust to incoming data streams, addressing distribution shifts between source and target domains. However, standard methods rely on single-dimensional linear classification layers, which often fail to handle diverse and complex shifts. We propose Hierarchical Adaptive Networks with Task Vectors (Hi-Vec), which leverages multiple layers of increasing size for dynamic test-time adaptation. By decomposing the encoder’s representation space into such hierarchically organized layers, Hi-Vec, in a plug-and-play manner, allows existing methods to adapt to shifts of varying complexity. Our contributions are threefold: First, we propose dynamic layer selection for automatic identification of the optimal layer for adaptation to each test batch. Second, we propose a mechanism that merges weights from the dynamic layer to other layers, ensuring all layers receive target information. Third, we propose linear layer agreement that acts as a gating function, preventing erroneous fine-tuning by adaptation on noisy batches. We rigorously evaluate the performance of Hi-Vec in challenging scenarios and on multiple target datasets, proving its strong capability to advance state-of-the-art methods. Our results show that Hi-Vec improves robustness, addresses uncertainty, and handles limited batch sizes and increased outlier rates. Code: https://github.com/ambekarsameer96/Hi-Vec

In this work, we introduce Progressive Growing of Patch Size (PGPS), an automatic curriculum learning approach for 3D medical image segmentation. Curriculum learning structures the training process by presenting progressively more complex samples to the model, often improving training convergence. In our case, we operationalize this by starting training with small patch sizes and gradually increasing them, which naturally improves the foreground-to-background class voxel ratio in early training stages. We evaluate our approach in two distinct settings. First, a resource-efficient mode maintains a constant batch size throughout training to reduce the input tensor size and computational cost (FLOPs) relative to conventional training. Second, a performance mode inversely scales the batch size relative to the patch volume, keeping the total FLOPs comparable to standard training while maximizing final segmentation quality. Both modes are evaluated on segmentation performance (Dice score) and computational costs across 15 diverse and popular 3D medical image segmentation tasks. The resource-efficient mode matches the segmentation performance of the conventional constant patch size baseline while reducing wall-clock training time to only 44%. We show that the performance mode improves upon the constant patch size baseline, achieving a statistically significant relative gain in mean Dice score of 1.28%. Remarkably, the performance mode surpasses the constant patch size baseline across all 15 tasks, while simultaneously reducing wall-clock training time to only 89%. We found that the benefits are particularly pronounced for tasks with severe foreground-to-background voxel imbalance, such as lesion segmentation. As a consequence of the improved convergence, the proposed performance mode reduces segmentation performance variance relative to conventional constant patch size training, making model comparisons less sensitive to training stochasticity. Finally, our experiments demonstrate that PGPS is not tied to a specific architecture but represents a broadly applicable strategy that consistently boosts performance across diverse segmentation models, including UNet, UNETR, and SwinUNETR. In summary, this simple yet effective transformation of the input sampling strategy substantially improves both segmentation performance and training efficiency, while remaining compatible with diverse segmentation backbones.

The sharp rise in medical tomography examinations has created a demand for automated systems that can reliably extract informative features for downstream tasks such as tumor characterization. Although 3D volumes contain richer information than individual slices, effective 3D classification remains difficult: volumetric data encode complex spatial dependencies, and the scarcity of large-scale 3D datasets has constrained progress toward 3D foundation models. As a result, many recent approaches rely on 2D vision foundation models trained on natural images, repurposing them as feature extractors for medical scans with surprisingly strong performance. Despite their practical success, current methods that apply 2D foundation models to 3D scans via slice-based decomposition remain fundamentally limited. Standard slicing along axial, sagittal, and coronal planes often fails to capture the true spatial extent of a structure when its orientation does not align with these canonical views. More critically, most approaches aggregate slice features independently, ignoring the underlying 3D geometry and losing spatial coherence across slices. To overcome these limitations, we propose TomoGraphView, a novel framework that integrates omnidirectional volume slicing with spherical graph-based feature aggregation. Instead of restricting the model to axial, sagittal, or coronal planes, our method samples both canonical and non-canonical cross-sections generated from uniformly distributed points on a sphere enclosing the volume. Triangulating these viewpoints yields a spherical graph that captures spatial relationships among views, and we use a graph neural network to aggregate their features accordingly. Experiments across six oncology 3D medical image classification datasets demonstrate that omnidirectional volume slicing improves the average performance in Area Under the Receiver Operating Characteristic Curve (AUROC) from 0.7701 to 0.8154 compared with traditional slicing approaches relying on canonical view planes. Moreover, we can further improve AUROC performance from 0.8198 to 0.8372 by leveraging our proposed graph neural network-based feature aggregation. Notably, TomoGraphView also surpasses large-scale pretrained 3D medical imaging models across all datasets and tasks, underscoring its effectiveness as a powerful framework for volumetric analysis and therefore represents a key step toward bridging the gap until fully native 3D foundation models become available in medical image analysis. We provide a user-friendly library for omnidirectional volume slicing at https://pypi.org/project/OmniSlicer.