Commercial Web search engines have to process user queries over huge Web indexes under tight latency constraints. In practice, to achieve low latency, large result caches are employed and a portion of the query traffic is served using previously computed results. Moreover, search engines need to update their indexes frequently to incorporate changes to the Web. After every index update, however, the content of cache entries may become stale, thus decreasing the freshness of served results. In this work, we first argue that the real problem in today's caching for large-scale search engines is not eviction policies, but the ability to cope with changes to the index, i.e., cache freshness. We then introduce a novel algorithm that uses a time-to-live value to set cache entries to expire and selectively refreshes cached results by issuing refresh queries to back-end search clusters. The algorithm prioritizes the entries to refresh according to a heuristic that combines the frequency of access with the age of an entry in the cache. In addition, for setting the rate at which refresh queries are issued, we present a mechanism that takes into account idle cycles of back-end servers. Evaluation using a real workload shows that our algorithm can achieve hit rate improvements as well as reduction in average hit ages. An implementation of this algorithm is currently in production use at Yahoo!.

Systems that use or serve multimedia data require timely access to data on hard drives. To ensure adequate performance users must either prevent overload of disk resources, or use real-time algorithms that rely on intricate knowledge of disk internals to meet deadline requirements. We have developed Hierarchical Disk Sharing (HDS) to allow disks to be fully utilized while sustaining a bandwidth reservation, without requiring detailed knowledge of the drive internals. HDS uses a hierarchy of token bucket filters to isolate disk access among clients and groups of clients, and to allow for reclaiming of unused bandwidth. We discuss the design of HDS and present our implementation in a Linux block device driver, demonstrating the effectiveness (and limitations) of this approach.

Demand for real-time capability in general-purpose systems is rising and as systems are retrofitted with scheduling features they become increasingly complex. To counter this trend we present the best-effort bandwidth server (BEBS), an aperiodic server for flexible and efficient support of best-effort applications in a real-time system. Recognizing that the responsiveness of a server depends on its period, and that not every best-effort task requires equal responsiveness, the algorithm adjusts its period based on run-time behavior of tasks. We created a prototype implementation of the system to demonstrate that it performs suitably as a general-purpose scheduler in comparison to Linux, and outperforms a common type of hierarchy used in existing general-purpose systems. The result is a system that integrates real-time scheduling with best-effort support, both simple and powerful enough to be used as the only scheduler in a general-purpose operating system.

Real-time systems are growing in complexity and real-time and soft real-time applications are becoming common in general-purpose computing environments. Thus, there is a growing need for scheduling solutions that simultaneously support processes with a variety of different timeliness constraints. Toward this goal we have developed the resource allocation/dispatching (RAD) integrated scheduling model and the rate-based earliest deadline (RBED) integrated multi-class real-time scheduler based on this model. We present RAD and the RBED scheduler and formally prove the correctness of the operations that RBED employs. We then describe our implementation of RBED and present results demonstrating how RBED simultaneously and seamlessly supports hard real-time, soft real-time, and best-effort processes.