Dynamic optimization relies on phase detection for two important functions (1) To detect change in code working set and (2) To detect change in performance characteristics that can affect optimization strategy. Current prototype runtime optimization systems (J. Lu et al) compare aggregate metrics like CPI over fixed time intervals to detect a change in working set and a change in performance. While simple and cost-effective, these metrics are sensitive to sampling rate and interval size. A phase detection scheme that computes performance metrics by aggregating the performance of individually optimized regions can be misled by some regions impacting aggregate metrics adversely. In this paper, we investigate the benefits and limitations of using aggregate metrics for phase detection, which we call global phase detection (GPD). We present a new model to detect change in working set and propose that the scope of phase detection be limited to within the candidate regions for optimization. By associating phase detection to individual regions we can isolate the effects of regions that are inherently unstable. This approach, which we call local phase detection (LPD), shows improved performance on several benchmarks even when global phase detection is not able to detect stable phases.

Optimization of a real world application BLAST is used to demonstrate the limitations of static and profile-guided optimizations and to highlight the potential of runtime optimization systems. We analyze the performance profile of this application to determine performance bottlenecks and evaluate the effect of aggressive compiler optimizations on BLAST. We find that applying common optimizations (e.g. O3) can degrade performance. Profile guided optimizations do not show much improvement across the board, as current implementations do not address critical performance bottlenecks in BLAST. In some cases, these optimizations lower performance significantly due to unexpected secondary effects of aggressive optimizations. We also apply runtime optimization to BLAST using the ADORE framework. ADORE is able to detect performance bottlenecks and deploy optimizations resulting in performance gains up to 58% on some queries using data cache prefetching.

Data prefetching via helper threading has been extensively investigated on simultaneous multi-threading (SMT) or virtual multi-threading (VMT) architectures. Although reportedly large cache latency can be hidden by helper threads at runtime, most techniques rely on hardware support to reduce context switch overhead between the main thread and helper thread as well as rely on static profile feedback to construct the help thread code. This paper develops a new solution by exploiting helper threaded prefetching through dynamic optimization on the latest UltraSPARC chip-multiprocessing (CMP) processor. Our experiments show that by utilizing the otherwise idle processor core, a single user-level helper thread is sufficient to improve the runtime performance of the main thread without triggering multiple thread slices. Moreover, since the multiple cores are physically decoupled in the CMP, contention introduced by helper threading is minimal. This paper also discusses several key technical challenges of building a lightweight dynamic optimization/software scouting system on the UltraSPARC/Solaris platform.