Homomorphic Encryption (HE) provides strong data privacy for cloud services but at the cost of prohibitive computational overhead. While GPUs have emerged as a practical platform for accelerating HE, there remains an order-ofmagnitude energy-efficiency gap compared to specialized (but expensive) HE ASICs. This paper explores an alternate direction: leveraging existing AI accelerators, like Google's TPUs with coarse-grained compute and memory architectures, to offer a path toward ASIC-level energy efficiency for HE. However, this architectural paradigm creates a fundamental mismatch with SoTA HE algorithms designed for GPUs. These algorithms rely heavily on: (1) high-precision (32-bit) integer arithmetic to now run on a TPU's low-throughput vector unit, leaving its high-throughput low-precision (8-bit) matrix engine (MXU) idle, and (2) fine-grained data permutations that are inefficient on the TPU's coarse-grained memory subsystem. Consequently, porting GPU-optimized HE libraries to TPUs results in severe resource under-utilization and performance degradation. To tackle above challenges, we introduce CROSS, a compiler framework that systematically transforms HE workloads to align with the TPU's architecture. CROSS makes two key contributions: (1) Basis-Aligned Transformation (BAT), a novel technique that converts high-precision modular arithmetic into dense, low-precision (INT8) matrix multiplications, unlocking and improving the utilization of TPU's MXU for HE, and (2) Memory-Aligned Transformation (MAT), which eliminates costly runtime data reordering by embedding reordering into compute kernels through offline parameter transformation. Our evaluation on a real single-host Google TPU v6e refreshes the SoTA Number Theoretic Transform (NTT) throughput record with up-to$1.43 \times$throughput improvement over WarpDrive on a NVIDIA A100. Furthermore, CROSS achieves$451 \times, 7.81 \times, 1.83 \times, 1.31 \times, 1.86 \times$, and$1.15 \times$higher throughput per watt than OpenFHE, WarpDrive, FIDESlib, FAB, HEAP, and Cheddar, respectively, establishing AI ASIC as the SotA efficient platform for HE operators. Code: https://github.com/EfficientPPML/CROSS.

The rapid progress of artificial intelligence (AI) has been largely driven by deep neural networks, yet their insufficient compositional reasoning abilities, limited robustness, and lack of explainability expose fundamental limitations for next-generation cognitive systems. Energy-Based Models (EBMs), adhering to System-2-style thinking, have recently shown remarkable performance across generative, compositional, and reasoning tasks, offering a pathway toward more robust and cognitively grounded AI. However, the system behavior of EBMs remains poorly understood. Their long sampling loops, strict execution data dependencies, model compositionality and heterogeneity make them highly inefficient on off-the-shelf hardware, creating significant barriers for scalable and real-time deployment. In this paper, we present the first comprehensive workload characterization and system study of EBMs. We first introduce a taxonomy that unifies all the diverse EBM algorithms (around twenty), then formalize their end-to-end runtime to expose key optimization opportunities, and experimentally profile representative models across CPU, GPU, and TPU platforms to understand their runtime breakdowns, memory behavior, computational operators, roofline model, algorithmic performance, and MCMC (Markov Chain Monte Carlo) sampling impacts. Our analysis reveals fundamental system bottlenecks unique to EBMs, including highly repetitive forward and backward neural operations, long and inefficient sampling trajectories, and severe hardware under-utilization caused by heterogeneous model components and operations. Furthermore, based on the experiments and analysis, we suggest optimization solutions across the algorithm, system and architecture levels, to improve EBM computing performance, efficiency, and scalability, aiming to pinpoint the challenges and future directions of EBM research.