Quantum Random Access Memory (QRAM) is a crucial architectural component for querying classical or quantum data in superposition, enabling algorithms with wide-ranging applications in quantum arithmetic, quantum chemistry, machine learning, and quantum cryptography. In this work, we introduce Fat-Tree QRAM, a novel query architecture capable of pipelining multiple quantum queries simultaneously while maintaining desirable scalings in query speed and fidelity. Specifically, Fat-Tree QRAM performs $O(\log (N))$ independent queries in $O(\log (N))$ time using $O(N)$ qubits, offering immense parallelism benefits over traditional QRAM architectures. To demonstrate its experimental feasibility, we propose modular and on-chip implementations of Fat-Tree QRAM based on superconducting circuits and analyze their performance and fidelity under realistic parameters. Furthermore, a query scheduling protocol is presented to maximize hardware utilization and access the underlying data at an optimal rate. These results suggest that Fat-Tree QRAM is an attractive architecture in a shared memory system for practical quantum computing.