For an extensive period, Vision Transformers (ViTs) have been deemed unsuitable for attaining robust performance on small-scale datasets, with WideResNet models maintaining dominance in this domain. While WideResNet models have persistently set the state-of-the-art (SOTA) benchmarks for robust accuracy on datasets such as CIFAR-10 and CIFAR-100, this paper challenges the prevailing belief that only WideResNet can excel in this context. We pose the critical question of whether ViTs can surpass the robust accuracy of WideResNet models. Our results provide a resounding affirmative answer. By employing ViT, enhanced with data generated by a diffusion model for adversarial training, we demonstrate that ViTs can indeed outshine WideResNet in terms of robust accuracy. Specifically, under the Infty-norm threat model with epsilon = 8/255, our approach achieves robust accuracies of 74.97% on CIFAR-10 and 44.07% on CIFAR-100, representing improvements of +3.9% and +1.4%, respectively, over the previous SOTA models. Notably, our ViT-B/2 model, with 3 times fewer parameters, surpasses the previously best-performing WRN-70-16. Our achievement opens a new avenue, suggesting that future models employing ViTs or other novel efficient architectures could eventually replace the long-dominant WRN models.

With the wide adoption of GPU and the explosion in data volumes, existing accelerator-centric systems require massive storage access. They adopt high-performance storage devices like NVMe SSDs to scale up single-node systems cost-effectively and leverage the CPU to manage these SSDs. However, they suffer from performance bottlenecks because of the high CPU OS kernel overhead and the CPU memory intermediated data transfer. To address this issue, GPU-initiated and GPU-managed SSD management is proposed to allow the GPU to fully manipulate SSDs: 1) direct data transfer from SSD to GPU memory (data plane) and 2) GPU-managed SSD control (control plane). This can potentially enable these GPU systems to fully leverage the SSD bandwidth. However, we still identify two severe issues. First, the GPU-management SSD control leads to low GPU Streaming Multiprocessor utilization. Second, it leads to the serial execution of SSD accesses with GPU computation, which slows down the overall computing task. To this end, we propose CAM, the first asynchronous GPU-initialized, CPU-managed SSD management for batching storage access. It 1) offloads the SSD control plane from GPU to CPU, thus maximizing GPU streaming multiprocessor utilization, and 2) adopts asynchronous user-friendly APIs that allow programmers to easily overlap GPU computation and SSD I/O operations while keeping a synchronous programming experience. As such, CAM enables us to achieve the best of two worlds: high performance and high programmability. The experimental results show that CAM can perform GNN model training, mergesort, and GEMM up to$\mathbf{1.84}\times, \mathbf{1.5}\times$, and$\mathbf{1.84}\times \mathbf{faster}$compared to the existing state-of-the-art GPU systems, while keeping high programmability.

Graph Neural Networks (GNNs) are widely employed in applications like recommendation systems, social network analysis, and fraud detection, but training large-scale GNNs is challenging due to its memory limitations. Existing systems face a trade-off between throughput and monetary cost: Distributed systems require expensive memory scaling, while single-machine out-of-core systems are limited by GPU/PCIe throughput. To this end, we propose Moment, a physical communication topology and data placement co-optimizer to enable high-throughput and low-cost GNN training in a single multi-GPU machine. Moment addresses communication contention and GPU load imbalance issues by modeling the physical topology as capacity-constrained directed graphs and formulating communication scheduling as a max-flow problem. It also introduces a data-distribution-aware knapsack algorithm for optimized data placement. Experimental results show that Moment outperforms out-of-core systems by up to 6.51 × and distributed systems by up to 3.02 ×, with only 50% monetary cost.

Abstract The aim was to analyse the application value of magnetic resonance imaging (MRI) in the injury evaluation of posterolateral corner (PLC) caused by sports, and the ant colony‐based crowding degree of fish swarm optimization algorithm for image segmentation optimization of MRI images. The crowding degree of fish swarm was introduced to construct the ant colony optimization algorithm (ACOA), and the maximum and minimum ant system (MMAS) and ant colony algorithm based on variation features (VF‐based ACA) were introduced to compare with ACOA. Besides, ACOA was applied to the MRI diagnosis of 98 patients with PLC injuries. Then the number of Degree I, Degree II, and Degree III in PLC patients were compared. The results showed that the iteration times and running time of the optimal solution were obtained by ACOA when the number of cities was 10, which were compared with those obtained by MMAS and VF‐based ACA with no great differences ( P > 0.05), and the iteration times and running time of ACOA were greater than those of MMAS and VF‐based ACA when the number of cities was 50 ( P < 0.05). There were 73 patients with PLC injuries involved 2 or more structures, 11 patients with PLC injuries only involved the lateral collateral ligament (LCL), and 7 patients with PLC injuries only involved the popliteal tendon (PLT). The injury rates of LCL, medial collateral ligament, anterior cruciate ligament (ACL), posterior cruciate ligament (PCL), and PLT were 87.02%, 51.33%, 72.45%, 41.75%, and 84.06%, respectively. The number of patients with PLC injuries combined with Grade I of LCL, PCL, ACL, PCL, biceps femoris tendon, and PLT was higher than that of patients with Grade II and III ( P < 0.05). In conclusion, ACOA was much better than MMAS and VF‐based ACA in dealing with complex work, which effectively improved the ergodicity of search and reduced the running time. There were more common in PLC injuries combined with LCL, PCL, ACL, PCL, and PLT, MRI images based on ACOA could clearly show the degree of ligament and tendon injuries.