The development of data analytics services has fueled many optimizations in data scans, and indexes are one of the most important techniques to improve scan efficiency. Meanwhile, block-based data organization has become standard practice in these services, providing an opportunity for more fine-grained index selection at the block level. However, today’s systems ignore data distribution differences among blocks and usually tune indexes over the entire database table, leading to unnecessary storage costs and potential degradation in query performance. To bridge this gap, we propose BISLearner, a fast, block-aware index selecting approach based on reinforcement learning. One major challenge lies in differentiating the data distribution among data blocks. To solve this problem, BISLearner maintains simplified histograms that represent the data distribution of each block. When a query is issued, BISLearner leverages the query predicate and histogram-based block summaries to generate a specific workload representation for each block. However, such block-aware workload representation leads to an excessive number of input features, resulting in a slow or even incorrect convergence of neural networks. Inspired by the human-learning process, where more attention is devoted to the important parts of data, we design an attention-based neural model to efficiently handle the high volume of input features caused by table partitioning and select the best-suited index combinations at the block level. Additionally, to handle the expensive search space caused by attribute combinations and data partitioning, we employ heuristic-based invalid action masking at the block level to accelerate the training process. Our evaluation using PostgreSQL and Greenplum database systems demonstrates BISLearner is able to reduce job completion time by up to 28.45% compared to its best counterparts.