Running a typical Bitcoin client (also called full node) needs more than 444 GB of disk space, considerable time, and computational resources to synchronize the entire blockchain, which is infeasible for resource-constrained devices. To address such concerns, the lightweight Bitcoin client proposed by Satoshi outsources most of computational and storage burdens to full nodes. Unfortunately, interacting with full nodes to query transactions leaks considerable information like addresses and transactions of lightweight client users. Thus, Bitcoin users that rely on lightweight clients are subject to de-anonymization, which defeats users privacy. Traditional schemes cannot support lightweight clients to query transactions from full nodes in an efficient yet privacy-preserving way. In this article, we propose a new efficient yet privacy-preserving transaction query scheme that specially targets the missing support for lightweight clients. We identify unique characteristics of the Bitcoin blockchain and craft a highly customized private information retrieval scheme called BIT-PIR to match the Bitcoin transaction query scenario and boost performances. Moreover, we customize a storage structure of the Bitcoin blockchain so that it further improves the query efficiency of our scheme. Finally, we develop a prototype implementation to demonstrate the feasibility of our proposed scheme.