Thread-Level Speculation (TLS) utilizes speculative parallelization to accelerate hard-to-parallelize serial codes on multi-cores. As the heterogeneous multi-core architecture is becoming ubiquitous, it presents an opportunity for TLS to reorganize little cores for the acceleration of these serial codes instead of a big core with similar or more area and power. However, previous TLS designs significantly suffer from extended hardware overhead and costly speculative forwarding. We present LitTLS, a lightweight TLS design with versioning caches to eliminate significant extended hardware overhead by storing versions in caches without speculative write buffers and memory undo-logs. Additionally, LitTLS introduces the Speculative Address Table, a novel component to accelerate speculative forwarding with a central structure to trace memory dependencies. Evaluations on four little cores show that LitTLS achieves an average performance speedup of 2.87× compared to a little core, outperforming a big core by 94% with similar area and less power. The extended area size is only 0.07 mm 2 , and the maximum increase in dynamic power consumption is limited to 0.3%, compared to four little cores.

Lightweight Convolution and Vision Transformer hybrid models have increasingly dominated the frontiers of deep learning (DL) on edge devices; however, to the best of our knowledge, no prior work has provided comprehensive evaluation on hybrid models’ performance and analyzed their characteristics by diving deep into the edge ecosystem with diversified modern DL inference engines and heterogeneous hardware. This paper proposes a comprehensive open-source benchmark suite,ETBench, to allow power-efficiency, performance and accuracy assessment for state-of-the-art (SOTA) hybrid models across 11 most widely-used DL engines deployed on diverse edge devices. After building ETBench that satisfies 6 design requirements proposed in our work, we conduct extensive experiments on 14 devices including 19 CPUs, 11 GPUs and 5 NPUs, and obtain benchmark results from all deployment scenarios (combinations of models, quantization formats, software engines, and hardware platforms). Valuable observations and insightful implications are finally summarized. For example, within current DL engines, the INT8 quantization is significantly underperformed in terms of accuracy and speed against FP16 for hybrid models. Overall, ETBench serves as a collaborative platform that assists model architects in better evaluating their models and makes it possible for future co-optimizations of DL engines and hardware accelerators.