Out-of-distribution (OOD) robustness is a desired property of computer vision models. Improving model robustness requires high-quality signals from robustness benchmarks to quantify progress. While various benchmark datasets such as ImageNet-C were proposed in the ImageNet era, most ImageNet-C corruption types are no longer OOD relative to today's large, web-scraped datasets, which already contain common corruptions such as blur or JPEG compression artifacts. Consequently, these benchmarks are no longer well-suited for evaluating OOD robustness in the era of web-scale datasets. Indeed, recent models show saturating scores on ImageNet-era OOD benchmarks, indicating that it is unclear whether models trained on web-scale datasets truly become better at OOD generalization or whether they have simply been exposed to the test distortions during training. To address this, we introduce LAION-C as a benchmark alternative for ImageNet-C. LAION-C consists of six novel distortion types specifically designed to be OOD, even for web-scale datasets such as LAION. In a comprehensive evaluation of state-of-the-art models, we find that the LAION-C dataset poses significant challenges to contemporary models, including MLLMs such as Gemini and GPT-4o. We additionally conducted a psychophysical experiment to evaluate the difficulty of our corruptions for human observers, enabling a comparison of models to lab-quality human robustness data. We observe a paradigm shift in OOD generalization: from humans outperforming models, to the best models now matching or outperforming the best human observers.

Benchmarking models is a key factor for the rapid progress in machine learning (ML) research. Thus, further progress depends on improving benchmarking metrics. A standard metric to measure the behavioral alignment between ML models and human observers is error consistency (EC). EC allows for more fine-grained comparisons of behavior than other metrics such as accuracy, and has been used in the influential Brain-Score benchmark to rank different DNNs by their behavioral consistency with humans. Previously, EC values have been reported without confidence intervals. However, empirically measured EC values are typically noisy - thus, without confidence intervals, valid benchmarking conclusions are problematic. Here we improve on standard EC in two ways: First, we show how to obtain confidence intervals for EC using a bootstrapping technique, allowing us to derive significance tests for EC. Second, we propose a new computational model relating the EC between two classifiers to the implicit probability that one of them copies responses from the other. This view of EC allows us to give practical guidance to scientists regarding the number of trials required for sufficiently powerful, conclusive experiments. Finally, we use our methodology to revisit popular NeuroAI-results. We find that while the general trend of behavioral differences between humans and machines holds up to scrutiny, many reported differences between deep vision models are statistically insignificant. Our methodology enables researchers to design adequately powered experiments that can reliably detect behavioral differences between models, providing a foundation for more rigorous benchmarking of behavioral alignment.

In light of the recent widespread adoption of AI systems, understanding the internal information processing of neural networks has become increasingly critical. Most recently, machine vision has seen remarkable progress by scaling neural networks to unprecedented levels in dataset and model size. We here ask whether this extraordinary increase in scale also positively impacts the field of mechanistic interpretability. In other words, has our understanding of the inner workings of scaled neural networks improved as well? We use a psychophysical paradigm to quantify one form of mechanistic interpretability for a diverse suite of nine models and find no scaling effect for interpretability - neither for model nor dataset size. Specifically, none of the investigated state-of-the-art models are easier to interpret than the GoogLeNet model from almost a decade ago. Latest-generation vision models appear even less interpretable than older architectures, hinting at a regression rather than improvement, with modern models sacrificing interpretability for accuracy. These results highlight the need for models explicitly designed to be mechanistically interpretable and the need for more helpful interpretability methods to increase our understanding of networks at an atomic level. We release a dataset containing more than 130'000 human responses from our psychophysical evaluation of 767 units across nine models. This dataset facilitates research on automated instead of human-based interpretability evaluations, which can ultimately be leveraged to directly optimize the mechanistic interpretability of models.

GPU-based raycasting offers an interesting alternative to conventional slice-based volume rendering due to the inherent flexibility and the high quality of the generated images. Recent advances in graphics hardware allow for the ray traversal and volume sampling to be executed on a per-fragment level completely on the GPU leading to interactive framerates. In this work we present optimization techniques that improve the performance and quality of GPU-based volume raycasting. We apply a hybrid image/object space approach to accelerate the ray traversal in animation sequences that works for both isosurface rendering and semi-transparent volume rendering. An empty-space-leaping technique that exploits the spatial coherence between consecutively rendered images is used to estimate the optimal initial ray sampling point for each image pixel. These can double the rendering performance for typical volumetric data sets without sacrificing image quality. The achieved speed-up allows for further improvements of image quality. We demonstrate an object space antialiasing technique based on selective super-sampling at sharp creases and silhouette edges which also benefits from exploiting frame-to-frame coherence.

Volume visualization using isosurface extraction is a well-researched topic. Research demonstrated that even for unstructured grids peak performances of millions of tetrahedra per second can be achieved by exploiting the parallel processing capabilities of modern GPUs. In this paper we present a hardware-accelerated solution that further improves the extraction performance. In contrary to existing approaches, our technique explicitly extracts the isosurface geometry in a fragment program by rendering only a single screen-sized quadrilateral. The extracted geometry is directly written to an on-board graphics memory object allowing for direct rendering without further bus transfers. Additionally, the geometry can be manipulated by shader programs or read back to the application for further processing. Examples and application scenarios are given that can benefit from our approach.

Emerging technologies in communications and information processing are having, and will continue to have, a significant effect on the way command and control (C2) is carried out on the battlefield of the future. One of these technologies is a packet-switched, internetwork, computer-communications system. This system concept allows for the interconnection of heterogeneous computers on heterogeneous networks, thereby providing for rapid, efficient, and reliable communications between processing elements-regardless of their physical location. This type of telecommunications architecture also allows automatic network reconfiguration when switching nodes or communications links fail. This permits system survivability in stressed environments. This communications system provides the means for passing time-critical information in a reliable and efficient fashion for purposes of C2. To achieve survivable C2, however, it is also necessary to integrate distributed software and data-processing technology with this telecommunications architecture technology. To achieve a survivable and effective C2capability, an experimental distributed command, control, and communications C3architecture has been evolved that blends these evolving communications and processing technologies with innovative concepts in tactical battle management. A joint Army/Defense Advanced Research Projects Agency (DARPA) Testbed, located at Fort Bragg, North Carolina, is being established to explore survivable C3concepts based on this architecture. This testbed, supported under the Army/DARPA Distributed Communications and Processing Experiment (ADDCOMPE), was started in late 1984. The ADDCOMPE testbed is leveraging the results of the earlier Army Data Distribution (ADDS) testbed also situated at Fort Bragg. However, ADDS primarily addressed communications; the ADDCOMPE program's emphasis is not only on communications but also on distributedprocessing environments (software and hardware) and distributed C2applications implemented over such an environment. In ADDCOMPE, the ADDS communications architecture is being taken as "available" technology but is being extended to include packet-switched, internetwork communications implemented over present-day Army communications systems. In this paper, an overview of the ADDCOMPE telecommunications architecture is presented and a discussion of the general philosophy for use of the ADDCOMPE testbed to support technology transfer to the Army is given. As this architecture is implemented, future papers will discuss its technical details, such as protocols used, throughput delay characteristics, and the like. Future papers will also provide technical discussions on the distributed-processing software (system and C2applications) developed for the testbed.