Simulation validation, demonstrating that a simulation accurately represents the system it studies, poses a particularly potent form of the oracle problem. A simulation must be validated for its results to be reliable, but often no oracle exists due to the nature of simulations. We propose taking the metamorphic testing (MT) approach from the realm of software verification and applying it for validating executable simulation models. By establishing pseudo-oracles based on metamorphic relations between parameters and behaviors within an executable model, we can create a methodical approach for validating simulation models. We propose an overall framework and guidelines to apply MT for simulation validation, with details for two prevalent simulation approaches: agent-based simulation models (ABM), and discrete-event simulation (DES) modeling. Through three case studies, we demonstrate the successful application of MT toward validating three different simulation models: an ABM of gossip propagation, an ABM of cancer, and a DES of airport check-in and security.

Cryptographic hash functions are security-critical algorithms with many practical applications, notably in digital signatures. Developing an approach to test them can be particularly difficult, and bugs can remain unnoticed for many years. We revisit the NIST hash function competition, which was used to develop the SHA-3 standard, and apply a new testing strategy to all available reference implementations. Motivated by the cryptographic properties that a hash function should satisfy, we develop four tests. The Bit-Contribution Test checks if changes in the message affect the hash value, and the Bit-Exclusion Test checks that changes beyond the last message bit leave the hash value unchanged. We develop the Update Test to verify that messages are processed correctly in chunks, and then use combinatorial testing methods to reduce the test set size by several orders of magnitude while retaining the same fault-detection capability. Our tests detect bugs in 41 of the 86 reference implementations submitted to the SHA-3 competition, including the rediscovery of a bug in all submitted implementations of the SHA-3 finalist BLAKE. This bug remained undiscovered for seven years, and is particularly serious because it provides a simple strategy to modify the message without changing the hash value returned by the implementation. We detect these bugs using a fully-automated testing approach.

This paper describes an approach to the specification and management of the agents and resources that are required to support the execution of complex systems and processes. The paper suggests that a resource should be viewed as a provider of a set of capabilities that are needed by a system or process, where that set may vary dynamically over time and with circumstances. This view of resources is defined and then made the basis for the framework of an approach to specifying, managing, and allocating resources in the presence of real-world complexity and dynamism. The ROMEO prototype resource management system is presented as an example of how this framework can be instantiated. Some case studies of the use of ROMEO to support system execution are presented and used to evaluate the framework, the ROMEO prototype, and our view of the nature of resources.

In this paper, we examine the potential benefits of Web proxy caches in improving the effective capacity of servers and networks. Since networks and servers are typically provisioned based on a high percentile of the load, we focus on the effects of proxy caching on the tail of the load distribution. We find that, unlike their substantial impact on the average load, proxies have a diminished impact on the tail of the load distribution. The exact reduction in the tail and the corresponding capacity savings depend on the nature of the workload and the percentile of the load distribution chosen for provisioning networks and servers-the higher the percentile, the smaller the savings. For workloads considered in this study, compared with over a 50% reduction in the average load, the savings in network and server capacity was only 20%-35% for the 99th percentile of the load distribution. We also find that while proxies can be somewhat useful in smoothing out some of the burstiness in Web workloads; the resulting workload continues, however, to exhibit substantial burstiness and a heavy-tailed nature. We identify one-time requests for large objects to be the limiting factor that diminishes the impact of proxies on the tail of load distribution. We conclude that, while proxies are immensely useful to users due to the reduction in the average response time, they are less effective in improving the capacities of networks and servers.

Existing Web proxy caches employ cache consistency mechanisms to ensure that locally cached data is consistent with that at the server. We argue that techniques for maintaining consistency of individual objects are not sufficient; a proxy should employ additional mechanisms to ensure that related Web objects are mutually consistent with one another. We formally define the notion of mutual consistency and the semantics provided by a mutual consistency mechanism to end users. We then present techniques for maintaining mutual consistency in the temporal and value domains. A novel aspect of our techniques is that they can adapt to the variations in the rate of change of the source data, resulting in judicious use of proxy and network resources. We evaluate our approaches using real-world Web traces and show that: (i) careful tuning can result in substantial savings in the network overhead incurred without any substantial loss in fidelity, of the consistency guarantees, and (ii) the incremental cost of providing mutual consistency guarantees over mechanisms to provide individual consistency guarantees is small.

In this paper, we examine the potential benefits of web proxy caches in improving the effective capacity of servers and networks. Since networks and servers are typically provisioned based on a high percentile of the load, we focus on the effects of proxy caching on the tail of the load distribution. We find that, unlike their substantial impact on the average load, proxies have a diminished impact on the tail of the load distribution. The exact reduction in the tail and the corresponding capacity savings depend on the percentile of the load distribution chosen for provisioning networks and servers—the higher the percentile, the smaller the savings. In particular, compared to over a 50% reduction in the average load, the savings in network and server capacity is only 20-35% for the 99th percentile of the load distribution. We also find that while proxies can be somewhat useful in smoothing out some of the burstiness in web workloads; the resulting workload continues, however, to exhibit substantial burstiness and a heavy-tailed nature. We identify large objects with poor locality to be the limiting factor that diminishes the impact of proxies on the tail of load distribution. We conclude that, while proxies are immensely useful to users due to the reduction in the average response time, they are less effective in improving the capacities of networks and servers.