In recent years, we have witnessed the development of centralized Just-in-Time (JIT) compilers serving many compiler-less Language Virtual Machine (LVM) clients at once. While centralized JIT compilation reduces CPU and memory overhead through global code caching and code reuse, it introduces additional latency between LVM clients and the compilation server. In this paper, we investigate opportunities to hide this latency. We explore how sequences of JIT compilation requests can be learned and predicted by the centralized JIT compiler. We train a long short-term memory (LSTM) network on JIT compilation requests and, during deployment, use it to speculatively compile and deliver native code to the LVM clients with minimal latency. We have implemented our novel scheme in a distributed WebAssembly runtime environment and evaluated it against several benchmark suites, including real-world applications from various application domains. We show that learning JIT compilation sequences is feasible, and prediction is effective in hiding JIT compilation latency. We demonstrate that centralized JIT compilation assisted by code caching, predictive compilation, and code delivery reduces JIT compilation latency by, on average, 1–2 orders of magnitude across six benchmark suites and additional complex, real-world applications.

Heterogeneous-ISA processor designs have attracted considerable research interest. However, unlike their homogeneous-ISA counterparts, explicit software support for bridging ISA heterogeneity is required. The lack of a compilation toolchain ready to support heterogeneous-ISA targets has been a major factor hindering research in this exciting emerging area. For any such compiler “getting right” the mechanics involved in state transformation upon migration and doing this efficiently is of critical importance. In particular, any runtime conversion of the current program stack from one architecture to another would be prohibitively expensive. In this paper, we design and develop Unifico, a new multi-ISA compiler that generates binaries that maintain the same stack layout during their execution on either architecture. Unifico avoids the need for runtime stack transformation, thus eliminating overheads associated with ISA migration. Additional responsibilities of the Unifico compiler backend include maintenance of a uniform ABI and virtual address space across ISAs. Unifico is implemented using the LLVM compiler infrastructure, and we are currently targeting the x86-64 and ARMv8 ISAs. We have evaluated Unifico across a range of compute-intensive NAS benchmarks and show its minimal impact on overall execution time, where less than 6% overhead is introduced on average. When compared against the state-of-the-art Popcorn compiler, Unifico reduces binary size overhead from ∼200% to ∼10%, whilst eliminating the stack transformation overhead during ISA migration.