Holography has immense potential for near-eye displays in virtual and augmented reality (VR/AR), providing natural 3D depth cues through wavefront reconstruction. However, balancing the field of view (FOV) with the eyebox remains challenging, constrained by the étendue limitation. Additionally, holographic image quality is often compromised due to differences between actual wave propagation and simulation models. This study addresses these by expanding the eyebox via multi-angle illumination, and enhancing image quality with end-to-end pupil-aware hologram optimization. Further, energy efficiency is improved by incorporating higher-order diffractions and pupil constraints. We explore a Pupil-HOGD algorithm for multi-angle illumination and validate it with a dual-angle holographic display prototype. Integrated with camera calibration and tracked eye position, the developed Pupil-HOGD algorithm improves image quality and expands the eyebox by 50% horizontally. We envision this approach extends the space-bandwidth product (SBP) of holographic displays, enabling broader applications in immersive, high-quality visual computing.